70
METR AI Coding Research Inconclusive Because Dev Participants Refused to Complete Tasks Without AI (metr.org)
submitted 8 months ago by to c/programming@programming.dev
23 comments fedilink hide all child comments
 

Selected developer quotes:

“I’m torn. I’d like to help provide updated data on this question but also I really like using AI!” — a developer from the original study early-2025 when asked to participate in the late-2025 study.

“I found I am actually heavily biased sampling the issues … I avoid issues like AI can finish things in just 2 hours, but I have to spend 20 hours. I will feel so painful if the task is decided as AI-disallowed.” — a developer from the new study noting selection effects when choosing what tasks to include in the study.

“my head’s going to explode if I try to do too much the old fashioned way because it’s like trying to get across the city walking when all of a sudden I was more used to taking an Uber.” — a developer from the new study noting selection effects when choosing what tasks to include in the study.

you are viewing a single comment's thread
view the rest of the comments
[–] 8 points 8 months ago*

Well, i agree, but AI isn't equal to an IDE. AI is dangerous, it will give credible results that look purposeful, but not necessarily meaningful. It's a reviewers and thus a maintainers nightmare.

Also, as others have said, the very real risk of skill atrophy and real imposters who can build something but cannot understand how it works.

And just at a major environmental cost.

  • permalink
  • fedilink
  • source
  • parent