you are viewing a single comment's thread
view the rest of the comments
[–] 1 point 1 day ago (3 children)

Sure, but adding adversarial loops, or, some complex addition to every prompt would increase the token usage massively, thus the cost.

So, why bother with that?

I could very well be wrong, but all indications I’ve seen have been that that the cost of the tokens needed to get something even vaguely acceptable is greater than the cost of just hiring someone, or a group, to do the job. Unless something needs to be done fast due to an immediate concern, and the cost is no object.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 1 point 1 day ago (2 children)

    Perhaps they run a local model, then tokens are essentially free.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 1 day ago (1 child)

    Would need a very beefy cluster, and that definitely isn’t free.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 23 hours ago*

    Well yes, I'm not saying it's a free lunch. The point is a large up-front investment buys you unlimited tokens in perpetuity for no more than the cost of electricity to power the cluster — which can be surprisingly cheap for UMA machines.

    I've swapped off API models entirely and have been able to do a lot more useful work since I don't have to worry about token cost or usage limits. Granted it's slower per task (less tk/s), but overall much more productive, and private.

  • source
  • parent