you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 3 days ago* (last edited 2 days ago) (7 children)

Preamble: I am a huge LLM/codegen skeptic, for a litany of reasons.

With that said, this honestly sounds like whoever did the prompting didn’t include something like this in the prompt, along with nuanced adversarial review loops:

unsafe blocks are strongly discouraged, and must be specifically justified, and survive antagonistic review by a rust language specialist agent and software security specialist agent. Part of the intent of this reimplementation is to leverage the strengths of the rust language itself, and rampant unsafe usage directly contradicts that.

  • source
  • parent
  • hideshow 7 child comments
  • [–] 3 points 2 days ago (1 child)

    "It can't be that stupid, you must be prompting it wrong."

    The ol' reliable.

  • source
  • parent
  • hideshow 1 child comment
  • No, I’m saying that there are ways to introduce feedback mechanisms and antagonistic review and specialist agents to improve the end result of what codegen crap gives you. It’s by no means perfect, but it’s a damn sight better than “make no mistakes”, and if the quality of the rust code is really that much of a shitshow, it’s probably not even being reviewed by a human before getting yeeted into prod, let alone the other things I outlined.

  • source
  • parent
  • [–] 1 point 2 days ago (3 children)

    Sure, but adding adversarial loops, or, some complex addition to every prompt would increase the token usage massively, thus the cost.

    So, why bother with that?

    I could very well be wrong, but all indications I’ve seen have been that that the cost of the tokens needed to get something even vaguely acceptable is greater than the cost of just hiring someone, or a group, to do the job. Unless something needs to be done fast due to an immediate concern, and the cost is no object.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 1 point 2 days ago (2 children)

    Perhaps they run a local model, then tokens are essentially free.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 2 days ago (1 child)

    Would need a very beefy cluster, and that definitely isn’t free.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 2 days ago*

    Well yes, I'm not saying it's a free lunch. The point is a large up-front investment buys you unlimited tokens in perpetuity for no more than the cost of electricity to power the cluster — which can be surprisingly cheap for UMA machines.

    I've swapped off API models entirely and have been able to do a lot more useful work since I don't have to worry about token cost or usage limits. Granted it's slower per task (less tk/s), but overall much more productive, and private.

  • source
  • parent