There's an amount of time and some situations I'll give benefit of doubt to before deciding someone is just stupid XD

That's probably at least generally true which is why they're supposed to have panels of experts (in the portfolios/departments like anything scientifie that requires them) and go out and meet people in different places from different backgrounds and talk to them to find out what's going on AND ACTUALLY LISTEN TO THEM (as a lot of politicians will go out to be seen talking to people and pretend they're listening but still suffer the delusions that they somehow magically know better than everyone else).

Other people are generally fine; friends and family aside we would barely notice "normal" people, might remember the particularly nice/kind strangers, the stupid ones just stick in our heads more because how the hell are some people actually that stupid.