Totally agree with you.
My partner is always hassling me when we travel about: "why you doing that? Hurry up I need to find a toilet! You got a picture of that. You never take pictures of me. That will look crappy etc".
One of the main reasons I travel to is feed inspiration into my photography passion, so having a non-photographer tagging along can be a challenge in itself.
Keep up the great work, and enjoy your passion :-)
RE: How to Travel With a Photographer: Get Used to Walking on Your Own