I do realize that logic is based on a worldview, namely empiricism. The other day, I ran into some guy telling me to prove reality and argued against the existence of the world. So yeah, there are some people that just deny empiricism as everything was only a perception to them, thanks Nietzsche!

When I am outside of that world, that doesn't make sense.

Not quite. I can very much talk about worlds that do not apply to me, ie. abstract worlds, metaphysical entities, possible worlds etc. For example, an atheist can discuss the capacity of God's omnipotence even if the atheist does not subscribe to the nature of God.

Returning to my p = p example, I do not know of any ideology that denies an object of its own identity, which is why I claim p = p to be an objective fact. Even supposing the denial of the existence of p in the first place, well then the empty set is still equal to the empty set.