user224 comments on AI-generated code contains more bugs and errors than human output

Technology

89,627 readers

13,793 active

3,105 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related news or articles.
Be excellent to each other!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
Check for duplicates before posting, duplicates may be removed
Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 3 years ago

moderators

[–] user224@lemmy.sdf.org 7 points 10 months ago (4 children)

It works well for recalling something you already know, whether it be computer or human language. What's a word for... what's a command/function that does...

permalink

fedilink

source

parent

hideshow 4 child comments

[–] frongt@lemmy.zip 9 points 10 months ago (2 children)

For words, it's pretty good. For code, it often invents a reasonable-sounding function or model name that doesn't exist.

permalink

fedilink

source

parent

hideshow 2 child comments

[–] Xenny@lemmy.world 1 point 10 months ago* (1 child)

It's not even good for words. AI just writes the same stories over and over and over and over and over and over. It's the same problem as coding. It can't think of anything novel. Hell it can't even think. I'd argue the best and only real use for an llm is to help be a rough draft editor and correct punctuation and grammar. We've gone way way way too far with the scope of what it's actually capable of

permalink

fedilink

source

parent

hideshow 1 child comment

[–] Flisty@mstdn.social 1 point 10 months ago

@Xenny @frongt it's definitely not good for words with any technical meaning, because it creates references to journal articles and legal precedents that sound plausible but don't exist.Ultimately it's a very expensive replacement for the lorem ipsum generator keyboard shortcut.

permalink

fedilink

source

parent

[–] ThirdConsul@lemmy.ml 1 point 10 months ago

According to OpenAis internal test suite and system card, hallucination rate is about 50% and the newer the model the worse it gets.

And that fact remains unchanged on other LLM models.

permalink

fedilink

source

parent