Get to a point in your walk in God where you know who you are in Christ, not what man tell you, but what God has said about you.
If man don't know the assignment that is on your life, if they don't understand in depth the call that is on your life, they can become a hindrance.
This is why it is important that you build your relationship with God, because what God say about you is what matters.
Not everyone can go with you because not everyone understand the assignment, the call that is on your life.
When you look in the Bible, it tells us that the Lord wouldn't do as much as he wanted to do because they quenched the Holy Spirit.
He was not welcomed in his own hometown.
You cannot please everybody.
You won't please everybody.
But don't let people stop you or hinder you from doing what God has called you to do.