Performing procedural tasks using a statistical model of our language will never be reliable. There’s a reason why we use logical and proscriptive syntax when we want deterministic outcomes.
post
As someone who works in tech, currently testing AI integration into healthcare EHR- the current state of AI is simply not safe for anything outside transcription- and even that is error prone without strict re-reading (not scanning!) for error correction.
The errors can be subtle but life threatening. I highly recommended against integrating it - but the most lazy providers were already using AI illegally for their notes so this was seen as a middle road.
Medical care and provider training in the USA is not ok right now, and getting worse. AI and misinformation is accelerating the decline.
TL;DR: Three Hard Truths About AI Agents After building 12+ production systems, here's what I've learned: -Error rates compound exponentially in multi-step workflows. 95% reliability per step = 36% success over 20 steps. Production needs 99.9%+. Context windows create quadratic token costs. -Long conversations become prohibitively expensive at scale. -The real challenge isn't AI capabilities, it's designing tools and feedback systems that agents can actually use effectively.
The TL;DR of the TL;DR is compounding expensive, error-prone results.
It sounds like one should be building deliberate AI workflows with extra checks (automated or human in the loop) that make careful and cost efficient incremental progress toward a measurable goal.
Sounds like hard work... we should just build 1,000,000 MCP servers instead. raises pinkie to cornee of mouth
Interesting. I have never seen the economic side of it being discussed outside of nvidia stock prices.
I keep having the same question: would it benefit to have a separate agent whose job was to error-check the first agent?
Here's the uncomfortable truth that every AI agent company is dancing around: error compounding makes autonomous multi-step workflows mathematically impossible at production scale.
Obvious problem is obvious.
all 11 comments