Are you the lease holder? If so you can simply deny it. Honestly personal boundaries and property should be respected and you shouldn't feel bad about protecting yourself. Giving a key to a random stranger seems rather odd.