Then all you'd need is collusion between a sufficient number of people to get someone you don't like convicted of any crime.

It's a question that is easy to pose and incredibly difficult to solve (if not actually impossible to solve). Which is why the world's collective judicial systems are struggling with it and have been struggling with it since the invention of judicial systems.

Ideally, courts should be able to convict every single guilty person while letting every single innocent person walk free. We also want all victims compensated fairly for the wrong that had been done unto them, while we want all scammers to get nothing.

The problem here is that we don't know everything and that circumstances differ, so that a solid binary yes-or-no decision is hard to make.

More evidence (and victim testimonies are evidence) helps for sure, but then again context needs to be taken into consideration.

If everyone says he did it, that's strong evidence. On the other hand there have been more than enough victims of wrongful mob "justice" as well (look up for example Drachenlord if you want a recent and really shocking one).

And of course, if money's on the table and it's a very famous person, the chances of someone just jumping on the bandwagon to score some easy money are not zero either.

On the other side, a large majority of legitimate rape cases have nothing but the testimony of a single victim.

All in all it would be disingenuous to put up a single number (even if this is 99%) of victim testimonies that should be enough for an automatic conviction or acquittal. And it's also unrealistic to expect a flawless conviction/acquittal record.

It's always a balance between convicting innocent people and acquitting dangerous criminals. And the balance needs to be somewhere and it's always wrong for the individuals who didn't get justice.