Related: https://hbr.org/2025/09/ai-generated-workslop-is-destroying-productivity

It's possible they're getting more stuff written and more emails sent, but the lower signal to noise ratio means this isn't actually helping.

I haven't seen anything yet. At work, the ones that praise AI the loudest are exceptionally highly correlated with the people who lack a good understanding of the core concepts. The ones that just float around cargo culting and looking busy by making noise.

That said, LLMs are still useful tools, that are highly misused. What they're useful for, is a lot less than most think. The user needs to be the expert. If they're not, they'd be better off reading a book on the matter (and how things are looking, it might have to be one written before LLMs came out).

In my experience it can really only competently do the 80% that takes 20% of the effort. Brainstorming and planning and gathering data - and even that you have to sift through for accuracy so maybe it saves 10% of the work in the best situations.

The biggest benefit I've gotten from AI is that it sometimes enables me to do some things I never would have been able to do with a small budget and a small team. But that didn't save me any time, maybe even required more time.

I have limited experience, but ChatGPT was great for getting me over a hump when I was writing PowerShell. I was at a dev shop at the time and most of them seemed to feel the same way.

Vibe coding is a dead end, and no replacement for actually knowing the job, but an LLM can give you ideas and inspiration when you're stuck.

One example; I was trying for a Google Sheets/Calendar integration. Very thin documentation and almost no examples online. Had ChatGPT write it. Of course it didn't function, but I picked out a couple of lines that got me finished. I had spent hours banging my head when I could have cut to the chase.

So how could we study effectiveness when that depends on how it's used?