- A LOT of this comes from elitist "old heads". Until the 2000's or 2010's, anime in America came from importing it into specialty shops. These formed a community of sorts, a unique subculture of anime enthusiasts. English dubs were rare, but if they existed they were often done very cheaply. Bad translation, bad direction, bad acting, bad editing, bad mixing even. Because it was not a large or lucrative enough market to justify putting money into.
There was also online piracy, which included the occasional fan dub. Which like... The passion of fans can be great for some things, but most anime fans lacked the skills fkd professional quality audio production.
And so for a loooong time English dubs got the reputation for being garbage. That changed a while ago, but some people just have a hard time adjusting their expectations.
- Gatekeeping. It is simply harder to watch things in a language you do not known with subtitles. That keeps out the visually impaired. It requires a lot more cognitive effort to watch, and takes your eyes off the actual animation that you are meant to enjoy. A lot of these old heads learned enough Japanese to be able to understand basic conversation.
In the pirate scene you also have fan subs, which is something that is much more accessible to anime fans than audio production. You need a lot of equipment and talent and expertise in different areas to create a dub. You just needed a computer and the ability to translate to make a sub. I don't mean to diminish the effort and skill it takes to translate, but dubbing requires all of those same skills PLUS a whole lot more, so it is harder.
This is not unique to anime. People in the US who watch foreign films in different languages have a similar subculture.
Personally, I like to get high and watch anime, but I find it's harder to read when I'm high. I don't mind watching sub's stuff occasionally, but I prefer dubs.
- Ignorance. It was not ONLY the English voice acting that was bad. A lot of cheaply made anime in the 80's, 90's, and 00's had terrible Japanese voice acting too. However, if you don't know much Japanese and are just reading the subtitles, you are not really in as good of a position to evaluate it. A lot of old anime fans love their anime, and felt outcast by American mainstream culture. They related more to Japanese culture and want to think its better. So they just assume the voice acting was better even when they aren't able to evaluate it themselves.
Edit: and here they all come to prove my point. A bunch of elitist gatekeepers telling me I'm wrong for enjoying media the way I want to enjoy it.