381
AI coders think they’re 20% faster — but they’re actually 19% slower (pivot-to-ai.com)
submitted 1 year ago by to c/techtakes@awful.systems
102 comments fedilink hide all child comments
 

podcast version
video version

you are viewing a single comment's thread
view the rest of the comments
[–] -5 points 1 year ago (10 children)

The N=16 keeps getting buried. Deliberate?

  • permalink
  • fedilink
  • source
  • hideshow 10 child comments
    •
  • [–] 21 points 1 year ago (2 children)

    Paragraph 2:

    METR funded 16 experienced open-source developers with “moderate AI experience” to do what they do.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 27 points 1 year ago

    ... and just a few paragraphs further down:

    The number of people tested in the study was n=16. That’s a small number. But it’s a lot better than the usual AI coding promotion, where n=1 ’cos it’s just one guy saying “I’m so much faster now, trust me bro. No, I didn’t measure it.”

    I wouldn't call that "burying information".

  • permalink
  • fedilink
  • source
  • parent
  • [–] [S] 16 points 1 year ago (5 children)

    this user has been removed for commenting without reading the article

    being from programming dot dev is just the turd on top

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 11 points 1 year ago (4 children)

    programming.dev: statistical sampling excellency (worst edition)

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 4 child comments
    •
  • [–] 10 points 1 year ago (3 children)

    programmers learned what N means in statistics and immediately realized that “this N is too small” is a cool shortcut to sounding smart without reading the study, its goals, or its conclusions. and you can use it every time N is smaller than the human population on earth!

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
    • load more comments (1 reply)