▲ 135 ▼ AI’s recursive self-improvement might not come so quickly after all (www.technologyreview.com) submitted 1 month ago by inari@piefed.zip to c/technology@lemmy.world 34 comments fedilink hide all child comments
[+] MangoCats@feddit.it -7 points 1 month ago (3 children) [removed by mod] permalink fedilink source parent hideshow 3 child comments replies: [–] FiniteBanjo@feddit.online 1 point 1 month ago* (last edited 1 month ago) (1 child) They Don't pass 4/5. They pass 0/5 because they are 80% (that number is way too optimistic btw) accurate to human output on every one of the five attempts. They also can't be forced to learn and retake the trial because they don't have any contextual awareness, they just guess the next word in a sequence. Even if a machine made 4 self edits sucessfully, it would be permanently disfigured by the one failure and no longer be capable of making good edits. permalink fedilink source parent hideshow 1 child comment replies: [–] Zos_Kia@jlai.lu 2 points 1 month ago That's cool but you're describing the models from 2 years ago and also not considering harnesses, which account for most of the progress of the last year or so. permalink fedilink source parent [–] Zos_Kia@jlai.lu 0 points 1 month ago You're entirely right. This won't replicate to other fields like writing and creative arts in general, but software engineering is just not that hard and can basically be brute forced with a good harness. It's a done deal and there is no world where people will write professional code by hand. I like it cause it really separates coding (the job) from coding (the art form). People will code by hand for aesthetic reasons just like people learn the violin instead of using a synth and we'll have a generation of lovingly crafted stuff. But boring software will be entirely automated, if not generated on the fly based on immediate needs. permalink fedilink source parent replies: [+] MangoCats@feddit.it 1 point 1 month ago (1 child) [removed by mod] permalink fedilink source parent hideshow 1 child comment replies: [–] Zos_Kia@jlai.lu 1 point 1 month ago it’s precisely because software engineering is hard (not difficult, but rigid - objective) that makes it a good fit for LLM agent execution Yes i think we're actually in agreement here. I said "hard" (not difficult) as a reference to "hard problems", a term that comes from complexity theory but is now commonly used to describe problems which can't be reduced to an algorithm or evaluated objectively, and thus can't readily be "solved". Squishy subjects like music and sociology are full of hard problems, while solid subjects like math and coding are full of easy problems. Now the change introduced by LLMs is that as long as a problem is "easy", it can no longer be so laborious as to be impossible. Every software problem is solvable, modulo the effort/computing power you can spend on it. Your job as a software architect / engineer is actually to find the easiest workable solution(s). You also get bonus point if your solution is average (standard, unsurprising etc...), which makes it particularly soluble in LLMs which, by definition, can only produce output that is within the distribution of their training set. As for relative difficulty, the “soft” fields have a very sliding scale for that with a lot of allowance given to newbies that isn’t accepted “at the highest levels” That's not where i would put the difference. If you take a field like music, the problem is that it can't "just work". A nostalgic song may move the masses today but you can't say "okay we've solved nostalgia let's get to serenity next". Soon enough you'll need a new nostalgic song and by definition it will be out of distribution. You can't find it in a high dimensional representation of past music, and, well, you can't train on future data, so there is no way an LLM finds it and recognizes it for what it is. with a high bar of standards required before publication and release, LLMs are a powerful tool that can accelerate many creative processes I still believe they'll never amount to much regarding artistic processes, and not just for the reasons i already mentioned. To make something good you need to sit with it and walk with it and spend some time in it doing all the tedious little tasks until it really feels like home and you can express yourself in it. You can't achieve that if a machine speedruns all the little tasks for you. permalink fedilink source parent load more comments (-2 replies)
[–] FiniteBanjo@feddit.online 1 point 1 month ago* (last edited 1 month ago) (1 child) They Don't pass 4/5. They pass 0/5 because they are 80% (that number is way too optimistic btw) accurate to human output on every one of the five attempts. They also can't be forced to learn and retake the trial because they don't have any contextual awareness, they just guess the next word in a sequence. Even if a machine made 4 self edits sucessfully, it would be permanently disfigured by the one failure and no longer be capable of making good edits. permalink fedilink source parent hideshow 1 child comment replies: [–] Zos_Kia@jlai.lu 2 points 1 month ago That's cool but you're describing the models from 2 years ago and also not considering harnesses, which account for most of the progress of the last year or so. permalink fedilink source parent
[–] Zos_Kia@jlai.lu 2 points 1 month ago That's cool but you're describing the models from 2 years ago and also not considering harnesses, which account for most of the progress of the last year or so. permalink fedilink source parent
[–] Zos_Kia@jlai.lu 0 points 1 month ago You're entirely right. This won't replicate to other fields like writing and creative arts in general, but software engineering is just not that hard and can basically be brute forced with a good harness. It's a done deal and there is no world where people will write professional code by hand. I like it cause it really separates coding (the job) from coding (the art form). People will code by hand for aesthetic reasons just like people learn the violin instead of using a synth and we'll have a generation of lovingly crafted stuff. But boring software will be entirely automated, if not generated on the fly based on immediate needs. permalink fedilink source parent replies: [+] MangoCats@feddit.it 1 point 1 month ago (1 child) [removed by mod] permalink fedilink source parent hideshow 1 child comment replies: [–] Zos_Kia@jlai.lu 1 point 1 month ago it’s precisely because software engineering is hard (not difficult, but rigid - objective) that makes it a good fit for LLM agent execution Yes i think we're actually in agreement here. I said "hard" (not difficult) as a reference to "hard problems", a term that comes from complexity theory but is now commonly used to describe problems which can't be reduced to an algorithm or evaluated objectively, and thus can't readily be "solved". Squishy subjects like music and sociology are full of hard problems, while solid subjects like math and coding are full of easy problems. Now the change introduced by LLMs is that as long as a problem is "easy", it can no longer be so laborious as to be impossible. Every software problem is solvable, modulo the effort/computing power you can spend on it. Your job as a software architect / engineer is actually to find the easiest workable solution(s). You also get bonus point if your solution is average (standard, unsurprising etc...), which makes it particularly soluble in LLMs which, by definition, can only produce output that is within the distribution of their training set. As for relative difficulty, the “soft” fields have a very sliding scale for that with a lot of allowance given to newbies that isn’t accepted “at the highest levels” That's not where i would put the difference. If you take a field like music, the problem is that it can't "just work". A nostalgic song may move the masses today but you can't say "okay we've solved nostalgia let's get to serenity next". Soon enough you'll need a new nostalgic song and by definition it will be out of distribution. You can't find it in a high dimensional representation of past music, and, well, you can't train on future data, so there is no way an LLM finds it and recognizes it for what it is. with a high bar of standards required before publication and release, LLMs are a powerful tool that can accelerate many creative processes I still believe they'll never amount to much regarding artistic processes, and not just for the reasons i already mentioned. To make something good you need to sit with it and walk with it and spend some time in it doing all the tedious little tasks until it really feels like home and you can express yourself in it. You can't achieve that if a machine speedruns all the little tasks for you. permalink fedilink source parent load more comments (-2 replies)
[+] MangoCats@feddit.it 1 point 1 month ago (1 child) [removed by mod] permalink fedilink source parent hideshow 1 child comment replies: [–] Zos_Kia@jlai.lu 1 point 1 month ago it’s precisely because software engineering is hard (not difficult, but rigid - objective) that makes it a good fit for LLM agent execution Yes i think we're actually in agreement here. I said "hard" (not difficult) as a reference to "hard problems", a term that comes from complexity theory but is now commonly used to describe problems which can't be reduced to an algorithm or evaluated objectively, and thus can't readily be "solved". Squishy subjects like music and sociology are full of hard problems, while solid subjects like math and coding are full of easy problems. Now the change introduced by LLMs is that as long as a problem is "easy", it can no longer be so laborious as to be impossible. Every software problem is solvable, modulo the effort/computing power you can spend on it. Your job as a software architect / engineer is actually to find the easiest workable solution(s). You also get bonus point if your solution is average (standard, unsurprising etc...), which makes it particularly soluble in LLMs which, by definition, can only produce output that is within the distribution of their training set. As for relative difficulty, the “soft” fields have a very sliding scale for that with a lot of allowance given to newbies that isn’t accepted “at the highest levels” That's not where i would put the difference. If you take a field like music, the problem is that it can't "just work". A nostalgic song may move the masses today but you can't say "okay we've solved nostalgia let's get to serenity next". Soon enough you'll need a new nostalgic song and by definition it will be out of distribution. You can't find it in a high dimensional representation of past music, and, well, you can't train on future data, so there is no way an LLM finds it and recognizes it for what it is. with a high bar of standards required before publication and release, LLMs are a powerful tool that can accelerate many creative processes I still believe they'll never amount to much regarding artistic processes, and not just for the reasons i already mentioned. To make something good you need to sit with it and walk with it and spend some time in it doing all the tedious little tasks until it really feels like home and you can express yourself in it. You can't achieve that if a machine speedruns all the little tasks for you. permalink fedilink source parent
[–] Zos_Kia@jlai.lu 1 point 1 month ago it’s precisely because software engineering is hard (not difficult, but rigid - objective) that makes it a good fit for LLM agent execution Yes i think we're actually in agreement here. I said "hard" (not difficult) as a reference to "hard problems", a term that comes from complexity theory but is now commonly used to describe problems which can't be reduced to an algorithm or evaluated objectively, and thus can't readily be "solved". Squishy subjects like music and sociology are full of hard problems, while solid subjects like math and coding are full of easy problems. Now the change introduced by LLMs is that as long as a problem is "easy", it can no longer be so laborious as to be impossible. Every software problem is solvable, modulo the effort/computing power you can spend on it. Your job as a software architect / engineer is actually to find the easiest workable solution(s). You also get bonus point if your solution is average (standard, unsurprising etc...), which makes it particularly soluble in LLMs which, by definition, can only produce output that is within the distribution of their training set. As for relative difficulty, the “soft” fields have a very sliding scale for that with a lot of allowance given to newbies that isn’t accepted “at the highest levels” That's not where i would put the difference. If you take a field like music, the problem is that it can't "just work". A nostalgic song may move the masses today but you can't say "okay we've solved nostalgia let's get to serenity next". Soon enough you'll need a new nostalgic song and by definition it will be out of distribution. You can't find it in a high dimensional representation of past music, and, well, you can't train on future data, so there is no way an LLM finds it and recognizes it for what it is. with a high bar of standards required before publication and release, LLMs are a powerful tool that can accelerate many creative processes I still believe they'll never amount to much regarding artistic processes, and not just for the reasons i already mentioned. To make something good you need to sit with it and walk with it and spend some time in it doing all the tedious little tasks until it really feels like home and you can express yourself in it. You can't achieve that if a machine speedruns all the little tasks for you. permalink fedilink source parent