Does it make him less relevant because I don't? Not sure.

I don't think it makes him less relevant and possibly in different circumstances, quite the opposite. A terrible parent who died long ago, can still be the most relevant person in the child's life, right? We have relationships with everyone and everything, even if (and often especially when) we don't have them in our lives. The relevancy of money comes to mind.

Maybe relevance isn't about frequency. Maybe it's about weight.

This makes sense! The weight of the relationship (present or absent) is what matters. I like that framing a lot.