you are viewing a single comment's thread
view the rest of the comments
[–] 3 points 1 day ago (2 children)

Yeah, that’s fair. If you have one problem and solve it with regex, now you have two problems.

Though in this case the monthly inference budget ran out causing a system outage. I was on call, so, getting a page off hours makes me dislike the llm approach even more haha

  • source
  • parent
  • hideshow 2 child comments
  • [–] 0 points 1 day ago (1 child)

    If being out of tokens causes an outage, you have architectural problems

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 1 day ago

    On the contrary. If you architect to not fail when missing context critical to customer experience your architecture is irrelevant because you're losing customers. The issue was the Layer8 decision to use inference when regex it's a better fit.

    But lol imagine making deep assumptions when you don't even know what the product is.

    Fail fast. Fail loud.

  • source
  • parent