Totally agree with you.
My partner is always hassling me when we travel about: "why you doing that? Hurry up I need to find a toilet! You got a picture of that. You never take pictures of me. That will look crappy etc".
One of the main reasons I travel to is feed inspiration into my photography passion, so having a non-photographer tagging along can be a challenge in itself.
Keep up the great work, and enjoy your passion :-)