LLMs on the boundary (deadsimpletech.com)
 

If you've spent any time on LinkedIn in the last year or so, you'll likely be quite familiar with a certain genre of post generally along the lines of "this new model changes everything. Everything you thought you knew about AI before this was wrong!" These have, of course, been a thing since at least 2022, and they belie a lack of understanding about what it is that makes these models less useful than they promise to be. The posts largely ignore the fact that much of the behaviour of interest to us is dependent not on the model, but on how the model is constrained, and that furthermore technology for constraining LLMs is not very good. Specifically, we have precisely one method of doing it: the prompt.

 

A fair amount has changed since last Wednesday, and Safari accounts for most of what matters here. Safari 27 can now keep scrolled content in place when something is inserted above it, follows several more HTTP caching rules, and lets Service Workers declare routes that bypass their own fetch handlers. Chrome DevTools has also completed its soft-navigation workflow and added reproducible CPU-tier controls. The common thread is browser behaviour that can change a user’s experience without a site deployment.

 

Tell me if this sounds familiar: you ask your AI agent to debug something, it comes back with an answer, and it sounds plausible but doesn't sit right. You ask it "Are you sure? This doesn't pass the smell test", and you know what comes next.

"You're absolutely correct!" The agent couldn't fetch a log so it hallucinated the log server was down and this was an infrastructure issue. The conversation compacted and suddenly conflated two issues. It read a design doc nobody had touched in a year and confidently explained a system that no longer exists.

view more: ‹ prev next ›