453
Lavalamp too hot (discuss.tchncs.de)
submitted 9 months ago by to c/programmer_humor@programming.dev
71 comments fedilink hide all child comments
 
you are viewing a single comment's thread
view the rest of the comments
[–] 118 points 9 months ago (23 children)

If you have ever read the "thought" process on some of the reasoning models you can catch them going into loops of circular reasoning just slowly burning tokens. I'm not even sure this isn't by design.

  • permalink
  • fedilink
  • source
  • hideshow 23 child comments
    •
  • [–] 9 points 9 months ago (16 children)

    Why would it be by design? What does that even mean in this context?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 16 child comments
    •
  • [–] 6 points 9 months ago (14 children)

    You have to pay for tokens on many of the "AI" tools that you do not run on your own computer.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 14 child comments
    •
  • [–] 8 points 9 months ago* (9 children)

    Hmm, interesting theory. However:

    1. We know this is an issue with language models, it happens all the time with weaker ones - so there is an alternative explanation.

    2. LLMs are running at a loss right now, the company would lose more money than they gain from you - so there is no motive.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 9 child comments
    •
  • [+] 3 points 9 months ago (1 child)
    [removed by mod]
  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 0 points 9 months ago*

    No, it wasn't a virtue signal, you fucking dingdongs.

    Capitalism is rife with undercooked products, because getting a product out there starts the income flowing sooner. They don't have to be making a profit for a revenue stream to make sense. Some money is better than no money. Get it?

    Fuck, it's like all you idiots can do is project your lack of understanding on others...

  • permalink
  • fedilink
  • source
  • parent
  • [–] -2 points 9 months ago (7 children)

    Of course there's a technical reason for it, but they have incentive to try and sell even a shitty product.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 7 child comments
    •
  • [–] 1 point 9 months ago (6 children)

    I don't think this really addresses my second point.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 6 child comments
    •
  • [–] 0 points 9 months ago (5 children)

    How does it not? This isn't a fucking debate. How would artificially bloating the number of tokens they sell not help their bottom line?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 0 points 9 months ago (4 children)

    Because they currently lose money for every token sold. They're operating at a loss to generate a userbase so that they can monetize later. They're currently in the pre-enshittification (I still don't like that word) phase where they want to offer a good product at a loss and lure in customers, not phase 2 where they monetize their userbase.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 0 points 9 months ago (3 children)

    and? How do you not understand that more money is better for them even if they're not in the black, yet?

    Two things can be true at once.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 0 points 9 months ago* (2 children)

    Creating additional tokens LOSES them money. For a single token, the cost of generating it exceeds the profits.

    I genuinely don't understand what would drive someone to be this condescending when you don't even understand the argument I have clearly laid out four times now.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 0 points 9 months ago (1 child)

    Do you think they don't want people using their product? Are you really that dense?

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 9 months ago

    Are you? Because now we've agreed on every fact to determine my conclusion is correct. Yes they do want people using their product; they want to lure in customers. Wasting tokens generating unhelpful output would both drive customers away with a worse experience, and cost them more money. So there's no reason for them to do that. Like I said in my first post.

  • permalink
  • fedilink
  • source
  • parent
    • load more comments (-1 replies)
  • [–] 1 point 9 months ago (3 children)

    Dont they charge be input tokens? E.g. your prompt. Not the output.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 4 points 9 months ago* (2 children)

    I think many of them do, but there are also many "AI" tools that will automatically add a ton of stuff to try and make it spit out more intelligent responses, or even re-prompt the tool multiple times to try and make sure it's not handing back hallucinations.

    It really adds up in their attempt to make fancy autocomplete seem "intelligent".

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 9 months ago (1 child)

    Yes, reasoning models... but i dont think they would charge on that... that would be insane, but AI executives are insane, so who the fuck knows.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 9 months ago*

    Not the models. AI tools that integrate with the models. The "AI" would be akin to the backend of the tool. If you're using Claude as the backend, the tool would be asking claude more questions and repeat questions via the API. As in, more input.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 4 points 9 months ago

    This kind of stuff happens on any model you train from scratch even before training for multi step reasoning. It seems to happen more when there's not enough data in the training set, but it's not an intentional add. Output length is a whole deal.

  • permalink
  • fedilink
  • source
  • parent