On the bar chart I included what I think are the most popular models right now (not objectively, based on my anecdotal observations, link to include/exclude what you want).
What do you think? Is it fair? There was, and to some digree still is, a bit of controversy about AA score indexes after they updated their benchmarks to V5.0. I am mainly interested in your thoughts about QFN in relation to other models.
Gemini 3.8 does not match its benchmarks.
It forgets system instructions, spends too much time thinking, and has this weird bug with reasoning token reprocessing that totally throws off the model.
Mistral Large should be higher up, it’s far more reliable than its benchmarks shows.
I don't know what they are testing but I use QFN for everything.
Assuming it doesn't have to rely on internal knowledge too much by giving it tools to retrieve information, it does everything I ask it to.
all 11 comments