Thinking logically, you can actually be selfish by abandoning it and then going out with a better man than your partner. You will not care what happens after you leave it. Even if he's looking for you, you'll try to keep him from finding you.
When he is vulnerable and asks you to come back, you will tell him that is the reward of what he has done to you. Once he regrets and apologizes to you, you will say that it is just a nonsense and too late to regret it all. Worse than that, you can just hope that he is no longer in front of you and gone for ever.