you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 14 hours ago (1 child)

As someone who doesn't know much about their paid tiers, if you host it on your hardware, are there really some that are worth paying for vs just using one that's available for free? I kind of assumed there wasn't much of a difference and that's why they were trying to ban ones who had "outside influence.". Because in the end they'll all be using mostly the same data sets and the differences will be who is manipulating the answers to influence the end users, which has some profit to it, but not nearly as much as they would need

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 12 hours ago

    I'm not an expert either. I mostly use Opus at work and I don't pay for it - the company does. I've noticed Fable almost seems like a waste for a lot of tasks, so mostly I don't use it. I've not tried dialing it down to Sonnet or even Haiku to see how things fare, but keep meaning to try that out.

    If you look at leaderboards like:

    https://llm-stats.com/leaderboards/llm-leaderboard

    You'll see Fable 5.2 is at the very top with 61.1. Even Opus is 58.7.

    Here, you can see the very top is 54.9 from Kimi and I'm not sure which of these you can host yourself. My understanding is that just because something is open weight doesn't mean you can download it and in some instances, even when you can, the hardware to run it is not something exactly consumer-class (but a company might choose to build or rent that hardware themselves).

    https://llm-stats.com/leaderboards/open-llm-leaderboard

    When you remember how people went gaga over this in 2022...imagine using a model that "lags" some of the top-tier stuff but applied to more mundane things - it's not like the capabilities from 2022 or whatever evaporated. Suppose you assume you can run something at decent enough speeds, but it "lags" the others by 6 months to a year?

    At some point, it just seems to me there will be diminishing returns to keep paying for the most expensive option. My company offers me Fable now and I don't find that I need it for day to day things. For all I know, Haiku might be enough for some things.

  • source
  • parent