you are viewing a single comment's thread
view the rest of the comments
[–] 75 points 3 weeks ago (32 children)

Assuming these updates are fully tested before they're deployed, by a human, it shouldn't be that impactful negatively that AI is being used. The problem comes when updates are moved into production without human oversight.

But it gives me the heebie jeebies, since AI is known to make and fuck shit up, and I'm stuck using MS Teams every day in my job so I see all the weird shit that vibe coding produces. I don't relish the thought of weird, stupid shit happening to my Linux installation.

  • source
  • hideshow 32 child comments
  • [–] 44 points 3 weeks ago* (last edited 3 weeks ago) (1 child)

    Plus studies on skill decay when using AI. Also the linux maintainer onboarding pipeline—which was already critically endangered—is now no longer building understanding through writing code, just monitoring it's generation.

    I personally think you need to write a language by hand without AI doing it for you to learn in detail. It's like trying to learn how to do math or how to write mandarin by just supervising an AI doing it... you need to do it yourself. Supervising is fine if you're already fluent in mandarin and have been writing it for years and years. If you're just learning it's less than useless. If anything its directly harmful to learning.

    Same with speaking a language yourself without crutching your native language as the best method to learn to speak it (immersion)

    Edit: The long term externalities of AI—like pretty much everything nowadays—is going to be made worse by short term thinking. Quarterly profits and yearly results take precedence over long term skill development. Even OSS like linux will eat its seed corn out of expediency because of a lack of maintainers and new candidates being onboarded into the ivory tower mailing lists...

  • source
  • parent
  • hideshow 1 child comment
  • [–] 32 points 3 weeks ago (21 children)

    Human review of AI is sufficient in theory, but in practice it's never as thorough as the understanding you get writing it yourself. It's hard to avoid AI, but it is a constant invitation to laziness.

  • source
  • parent
  • hideshow 21 child comments
  • [–] 19 points 3 weeks ago (19 children)

    Unlike reviewing a human who writes code there is zero chance of AI learning from the past and making fewer mistakes in the future. At a least a human has a chance of becoming better at coding from review feedback.

  • source
  • parent
  • hideshow 19 child comments
  • [+] -16 points 3 weeks ago (17 children)

    No? Are you not aware of the concept of "memories"? AI do indeed take feedback into account, if they're designed to.

  • source
  • parent
  • hideshow 17 child comments
  • [–] 18 points 3 weeks ago (14 children)

    AI does not have memories.

    It has a comparatively small context window that makes things worse the more it fills up.

  • source
  • parent
  • hideshow 14 child comments
  • [+] -12 points 3 weeks ago (11 children)

    You are mistaken. Some AI models do indeed have memories.

  • source
  • parent
  • hideshow 11 child comments
  • [–] 10 points 3 weeks ago (10 children)

    They save some strings in a document to later read and "remember". Calling them memories is a reach.

  • source
  • parent
  • hideshow 10 child comments
  • [+] -6 points 3 weeks ago (9 children)

    My point stands that they can learn from feedback. Your memories are only information in a virtual document, too. Just as susceptible to corruption, or misinterpretation.

  • source
  • parent
  • hideshow 9 child comments
  • [–] 4 points 3 weeks ago (6 children)

    Does an Alzheimer's patient learn things just because they read the same note every morning explaining the reality to them?

  • source
  • parent
  • hideshow 6 child comments
  • [–] 0 points 3 weeks ago (5 children)

    No, their file system is corrupt. No reliable read or write access.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 1 point 3 weeks ago (4 children)

    The AI is no different from the girl in "50 First Dates (2004)". It has to go back and read the post-it notes every time you start a new session. It does not retain memory without writing shit down to read later.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 1 point 3 weeks ago (3 children)

    You're not wrong. It's an inferior equivalent, but would you not say she learns with each note she makes, hampered though she is?

  • source
  • parent
  • hideshow 3 child comments
  • [–] 1 point 3 weeks ago (2 children)

    Yes I would say that she temporarily learns something while reading the notes. And she stores it in short-term memory.

    The difference with AI though is that it has to re-read the same note again and again after each question you ask it. It does not keep things in "memory" from your previous conversations. It reads it again before answering you.

    That is why context length matters and is important in LLMs. There's a huge difference between talking to an AI with 10k in context length and 1mil in context length. You can try it yourself using Llama.cpp and running LLMs locally.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 2 weeks ago* (last edited 2 weeks ago) (1 child)

    Our short term memory is our context window. Some people have larger capacity for information, others very little. Same thing, different terminology.

    I use Ollama daily. The context limit of 10k is indeed very limiting and noticeable compared to the extended lengths of cloud models. I've had to recruit a model specifically trained on condensing salient information to perform context compression frequently. I also use the limited local context to establish effective strategies for maximizing token efficiency in the agent.md.

    It's no different from how many tasks a person can simultaneously perform, or disparate words they can hold concurrently. Eventually thoughts are forgotten. What exactly is determined by the structure itself - the context-pruning strategy inherent. We don't know what our own are, but I believe we have an equivalent.

    All I'm saying, in the end, is that anything that can occur in a physical entity can be emulated in a digital entity. Are we capable of doing that flawlessly yet? No, but it is looking inevitable to me. Neural networks are ubiquitous, and our ability to simulate them will only improve.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 2 weeks ago

    Sure, but I still think calling it "memories" is a reach and a bit disingenuous. It's not remembering anything. It's re-reading it will the every run, making it appears as if it has remembered. Calling it "reminders" or something else I think would me more appropriate.

    The "memories" terminology comes from the big AI companies and they are doing everything in their power to make AI seem smarter than it actually is. A lot of reporters in media has a hard time understanding this as well and will report on AI as having it's "own thoughts" and "making own decisions".

    ChatGPT breaches confinement because their engineers are stupid. Not because of some self determination to fuck with us or our infrastructure. OpenAI are responsible and let it happen. Same with Anthropic.

    In the end an AI is a dumb computer program that only know and does what we tell it to do.

  • source
  • parent
  • [–] 9 points 3 weeks ago (1 child)

    The auto-generated memories I've seen it create are pretty inane: "always name variables container_ when in MySpecificClass". Not really learning engineering principles.

    Similar, I've found AGENT.md files might nudge it in the right direction, but as always it's a fuzzy matcher, not a learning engineer.

  • source
  • parent
  • hideshow 1 child comment
  • [+] -8 points 3 weeks ago

    It's best to generate memories manually, but I've had good fortune with auto-generated so far. At least using Co-pilot.

    I have found it best to have another AI generate the agent.md - having a layer of interpretation can really clarify the information imparted.

  • source
  • parent
  • [–] 22 points 3 weeks ago (5 children)

    One thing I'm noticing about AI code is it's very, very task oriented. Like it will produce code, that works for the one objective, but absolutely will not account for other interactions with other parts of the system. It's really bad at juggling multiple priorities so for every solution it creates, it causes another 2 problems with other parts of the stack. Human coders would design a solution that takes those interactions into account from the ground up. With an AI solution, you kind of have to rebuild those interactions as an afterthought, which is where a lot of failures occur. Since interaction of multiple subsystems is kind of the whole point of a kernel... I have concerns.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 3 points 3 weeks ago (4 children)

    Human coders would design a solution that takes those interactions into account from the ground up.

    Lol, this only happens in Utopia

  • source
  • parent
  • hideshow 4 child comments
  • [–] 13 points 3 weeks ago (3 children)

    Utopic might be a bit of a stretch, but I should probably qualify that as good human coders.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 7 points 3 weeks ago* (1 child)

    it shouldn't be that impactful negatively that AI is being used

    Doesn't it invalidate the GPL? Open source licenses depend on copyright, and LLM-generated code isn't copyrightable.

    (Also, it can't be proven not to contain plagiarised proprietary code, in fact it's quite likely that it does; between this, the testing / review burden, and the mental decay of developers that use LLMs instead of coding, this pretty much kills open source projects, in several different ways to make sure they stay dead...)

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 3 weeks ago

    It's not that they're not copyrightable. That court case is misunderstood. The vibe coder tried to give the AI the copyright and the judge ruled that was stupid because AI can't own shit.

    Also copyright doesn't mean much anymore because companies are rewriting open source software as copyrightable vibe coded bullshit the same way people are "open sourcing" Adobe using vibe coded crap.

    It's garbage "progress" all the way down.

  • source
  • parent