Well, again, many groups have has the scientific establishment lie to them in order to experiment on them. I can't really blame people who have that in their cultural memory for being skeptical of the current scientific establishment too. I can't view that as stupid. For example, even if the Nazis have the scientific establishment backing them up (they did in some cases), I don't want people trusting them. I don't think that's wrong. It's often hard to impossible to separate science and the state.
I don't think that's happening today though, at least not to a large extent. I think science is a lot more open now, and there's too many people watching for the same things to happen without us knowing. There are probably some pretty fucked up small experiments happening today, but not on the scale of vaccines. However, I feel it's important to see where people are coming from, especially if you want to convince them of something. If you want to convince marginalized groups to trust the concensus on vaccines, you need to understand why they'd be skeptical so you can overcome that skepticism.