I think the fundamental issue is that "works" doesn't have a good measurable metric and so when discussing it tends to fall into that false binary that you correctly identified.
The best I've seen that attempts to work around this problem was this paper from back in 2014^1^. Unfortunately their results showed that while you're correct that causitive impact is not zero that <5% correlation, especially for a field with as high a signal/noise ratio as political science, is an incredibly disheartening answer for "how much can voting accomplish?"
So while you are likely correct that it's not nothing, it does suggest reality is much closer to the meme than your attempt at "nuance".
If you have any sources that cite measurable and non-anecdotal impact that tell a different story I'd love to read them.
^1 linking the preprint because it's not paywalled^