all 82 comments

sorted by: hot top controversial new old
[–] 87 points 2 months ago (1 child)
  • [–] 60 points 2 months ago (4 children)
  • [–] 19 points 2 months ago (1 child)

    I've had it on my todo for years to work through ddc and trusting trust.
    Which is a method to verify a compiler is matching its source and thus trustworthy.

    An orthogonal approach is reproducible builds, which among many benefits can make sure a few people verifying things benefit everyone who can then see they have the same verified binaries.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 30 points 2 months ago (5 children)
  • [–] 6 points 2 months ago (2 children)

    The ending is rather unsatisfactory.

  • source
  • parent
  • hideshow 2 child comments
  • [+] [S] 3 points 2 months ago (1 child)
  • [–] 4 points 2 months ago

    Sounds like quitter talk to me. It's not like AI is an evolutionary process that just happens somewhere, it's a localized tool - and even if it spreads itself, that would be by far the most humonguous virus ever. I think those authors just write down their own fears.

  • source
  • parent
  • [–] 16 points 2 months ago (8 children)

    Maybe? If you poison the prompt then there's evidence and it can be undone. Poison fragments of the source training data, however, and that's some KT shit right there. Enterprise foundation models cost bonkers money to train and pretty much slurp up all the data on the internet for mostly automated annotation. Stick something in an obscure part of the internet which becomes part of the training and produces the malicious response and it's going to be both hard and expensive to detect or correct.

  • source
  • hideshow 8 child comments
  • [–] 6 points 2 months ago (5 children)

    except for poison to take in, it should be a pretty significant part of the dataset. Also, ngl, i'm not much informed on the topic, but aren't all the datasets, if we're talking about generic diffusion models and LLMs, already been formed? From what i gather, the innovation in AI mainly comes from utilizing new architectures, rather than training a model on something unique.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 8 points 2 months ago (4 children)

    The datasets are constantly expanding as new content is generated online. There's a degradation issue currently where the models are training on incorrect data generated by previous iteration of their own or other models and effectively poisoning itself to more confidently give the same incorrect information in future.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 2 points 2 months ago* (1 child)

    i've heard of the dataset poisoning and degradation caused by llm-generated content present in the dataset myself, but i'm not sure whether it was a practical observation, or a mere experiment. And I still fail to see how new datasets are really useful for developing a new llms, or how it's a problem for the devs to switch back to the older datasets.

    And the cornerstone stays the same: to have any significant effect on the final LLM quality, shouldn't the poisoned (either by llm-produced content, or by intentional poisoning) data portion be... well, statistically significant?

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 2 months ago* (last edited 2 months ago)

    One thing to keep in mind is that, when it comes to LLMs, the models have not significantly changed in architecture.

    There's been new experiments and advancements in architecture on neural networks, and machine learning for specific applications. But LLM, as they are being commercialized by AI corporations to the general public, have stayed relatively the same. Except for one thing. Increasing in size. Larger datasets, or more specialized datasets like with coding, and larger number of tokens in memory. This is why it takes such large data centers. It's all been just brute forcing greater capabilities by enlarging the models.

    One of the things with LLM is that all the dataset influences the weighs and probabilities of the results. Even if the dataset includes a single event of a chain of words (think of the pizza with superglue incident), it can show up in the results eventually.

  • source
  • parent
  • [–] 1 point 2 months ago (1 child)
  • [–] 44 points 2 months ago (6 children)

    The very fact that Anthropic is now injecting a kind of watermark into every output, is solid proof that such a Ken Thompson hack is a inevetable risk

  • source
  • hideshow 6 child comments
  • [–] 16 points 2 months ago (5 children)

    Ugh. Fuck "AI" and fuck Anthropic. But please read how the "watermarks" work. TL;TR its like a seeded run in a video game. With the seed and pseudo rng, you get the outcome i.e. the extruded text. In the watermark its the reverse, outcome + prng = seed. The result will be the same "quality" extruded garbage as before.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 1 point 2 months ago (2 children)

    A bit of a late response, but why are they mutually exclusive? I get that the "seed" can be extracted if you check for it, but who says that seed can't be somrthing nefarious? Like, someone vibecodes a website. You visit the website, extract the seed and it shows you the complete userbase of that website.

    I'm just sptiballing here though. It might not be the best example. I get where you're coming from, but why we cant we both be right.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 10 points 2 months ago (1 child)

    Time to get back to Assembly, I guess.

    But do you trust the CPU?

    Well, fuck

  • source
  • hideshow 1 child comment
  • [–] 22 points 2 months ago (1 child)

    Devs should be "dev managers and executives". Real developers know LLMs are basically just a tool for finding examples and helping with syntax. Sure they're useful, but I'd never let them write code, much less compile it. Who knows what they'd inject into a build.

  • source
  • hideshow 1 child comment
  • [–] 4 points 2 months ago

    Real developers know LLMs are basically just a tool for finding examples and helping with syntax.

    Your words on gods ears. I also found it to be reasonably useful when you're stuck in the documentation of a library clearly written by and for people who already know it.

    Unfortunately people who do not understand code are blissfully ignorant at what garbage they have the machine spit out. Of course until the LLM nukes their whole project folder or even disk because the probability engine unfortunately picked "clean slate" as the most probable next thing. Not as if that ever happened at companies, lol.

  • source
  • parent
  • [–] 6 points 2 months ago

    I think about this every day.

  • source
  • [–] 11 points 2 months ago* (last edited 2 months ago) (1 child)

    At work, I use AI for some things. Right now I'm rewriting some legacy spaghetti code that's had a bunch of things hacked into it over the years. I spoke to the person most familiar with the expected behaviour and used AI to combine his info plus the existing code and unit/integration tests into a list of requirements.

    I wrote the new code and tests based on the requirements rather than based on the old code. After each commit, I used AI to check for parity between the old and new code, and it keeps a Google Sheet up to date with the progress (which features were fully implemented, and which ones were missing or had gaps). I had AI write some tests cases too - given the list of requirements, write integration tests for them based on the style of a few tests I wrote by hand.

    It has some quirks (eg for tests it loves over-mocking even though our skills tell it to mock as little as possible) but it definitely speeds things up.

    I use AI for small side projects at work too. Tweaking and adding features I want to shared libraries, internal tools to help our team debug stuff and automate triaging of bug reports (they're all still reviewed by a human), etc.

    The entire reason I can trust its code is because I can read it and tweak it myself. I sometimes need to go through a few iterations to get AI code into an acceptable state. AI writing machine code directly, like what's been talked about recently and what this post is referencing, is such a dumb idea.

    There's other people at work that use AI for absolutely everything. Writing code, reading code, writing posts in our internal groups, etc. That's something I don't understand. Some people that are all-in on AI produce so much low-quality AI slop.

  • source
  • hideshow 1 child comment
  • [–] 2 points 2 months ago

    Yeah, I'm in a similar camp. AI is very good at dummy data and unit tests, fairly good at log traces and making fiddly changes across an entire codebase, somewhat good at boilerplate and making derivative code based on something similar, and somewhere between "passable" and "awful" at everything else. I use it for what it's good at with heavy manual review, and just about the only time I run with the first thing it spits out without looking it over is if it's something that is never intended to leave my machine.

    I can also usually sift the usable from the not-usable in the automatic PRs that our company has turned on. In the past couple of months, the ratio of good comments to bad ones has gotten slightly better; from about 20% good to about 55% good.

    But I'd never let an AI write a team message or acceptance criteria or anything. That's a human job.

  • source
  • parent
  • [+] 6 points 2 months ago (1 child)
  • [–] 3 points 2 months ago

    Mmh… but this concept only fully works on Open-Source hardware, doesn't it? Otherwise the microcode or CPU itself could still be an attack vector to infect the bootstrapped compiler?

    (Genuinely curious, I have no clue)

  • source
  • parent
  • [–] 5 points 2 months ago
    [–] 9 points 2 months ago (6 children)

    What does that even mean. Whoever said that just uttered some empty but smart sounding catch phrase. Such is all the talk about the wonders of Ai

  • source
  • hideshow 6 child comments
  • [–] 8 points 2 months ago (3 children)

    I think it's supposed to be that how AI turns high level instructions into code is compared to how compilers turn code into assembly. Implying that using AI is just a natural extension of the handing off work to the computers that we've already been doing.

    I wonder if you gave different AI models some c++ or something and told them to write assembly based on it how well they would do compared to an actual compiler

  • source
  • parent
  • hideshow 3 child comments
  • [–] 3 points 2 months ago* (2 children)

    I highly doubt they would manage to make AI spout out good assembly, they fuck up with high level code, imagine assembly

  • source
  • parent
  • hideshow 2 child comments
  • [–] 2 points 2 months ago (1 child)

    With sufficient (usually stolen) training data they could probably do it to a state where it's kinda usable in an assistive manner.

    Though since these LLMs often struggle with anything that doesn't exist in truly abhorrent abundance in its training data as well as the actual engineering part (i.e. structuring the code sensibly and consistently, which I'd wager is crutially important with Assembly to not end up with spaghetti) trying to have it generate something from scratch would probably end up in a huge mess.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 11 points 2 months ago (1 child)

    Linus Torvalds

    Not exactly, but something along those lines.

    We are talking about a guy still using mailing lists and patch files to conduct development on one of the largest codebases in the world, not exactly someone who jumps on any new shiny thing just to sound smart.

    https://thenewstack.io/torvalds-ai-programming-productivity/

  • source
  • parent
  • hideshow 1 child comment
  • [–] 8 points 2 months ago

    just to clarify, he'd not so much called LLMs "the new compilers" as he compared both to each other in a sense that an LLM is just another layer of analysis tooling between the developer and the final machine code, which, IMO, sounds much more reasonable than calling LLMs "the new compiler".

  • source
  • parent
  • [–] 4 points 2 months ago (30 children)

    It's a tool, use it where it works and don't where it doesn't.

  • source
  • hideshow 30 child comments
  • [–] 26 points 2 months ago* (7 children)

    That saying (or, dare I say, thought-terminating clichΓ©) glosses over the consideration of risks and costs that result from its uncritical usage. No sane person should trust it without serious reservations just because it fits the purpose, and that's true whether or not it bears the latest combination of letters peddled by tech bros. It's a dangerous, irresponsible mentality. It's a tool the same way a sledgehammer made out of plutonium is a tool.

  • source
  • parent
  • hideshow 7 child comments
  • [–] 8 points 2 months ago (5 children)

    Plutonium sledgehammer sounds fucking awesome though

  • source
  • parent
  • hideshow 5 child comments
  • [–] 8 points 2 months ago (4 children)

    Pretty effective because of its high density, too. It's a shame it's brittle and would give you heavy metal poisoning.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 5 points 2 months ago (2 children)

    As ways to die go that also sounds pretty rad

  • source
  • parent
  • hideshow 2 child comments
  • [–] -3 points 2 months ago

    I mean in an old nuclear bomb that's pretty much the situation. Use a conventional explosive to slam some plutonium like a sledgehammer and nukes have proven to be a strong deterrent when a country had them. Still a tool with a use case

  • source
  • parent
  • [–] 6 points 2 months ago (2 children)

    If a tool costs more to be made than the equivalent of buying every single person on earth a loaf of bread every single day for 300 days, only for it to have niche uses, that only helps what is possible with already existing tools, is it worth it?

  • source
  • parent
  • hideshow 2 child comments
  • [–] 6 points 2 months ago (3 children)

    People probably say that about asbestos too.

  • source
  • parent
  • hideshow 3 child comments
  • [–] -2 points 2 months ago (1 child)

    Of course, it still has uses. Just not in home insulation.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 1 point 2 months ago

    Your comment is so fit for purpose, displaying such a massive willful ignorance about the world in general. Because the biggest problem with asbestos wasn't using it for home insulation. It killed millions of workers in manufacturing plants and mines. Its biggest industrial use case was for electrical insulation of high voltage transmission lines. Where it also killed hundreds of thousands of technicians in the us alone. Before that it was used for lamp wicks. Where it killed at least a couple of millions more over a century. Today, asbestos kills 250 thousand people a year, 12 to 15 thousand in the us every year.

    I supposed any tool can be considered useful if you're ignorant enough about it or if you're willing to lie without remorse.

  • source
  • parent
  • [–] 3 points 2 months ago (13 children)

    Na, fuck that shit. Just don't use it.

  • source
  • parent
  • hideshow 13 child comments
  • [–] 2 points 2 months ago (12 children)

    Why? Do you have that feeling towards other algorithms?

  • source
  • parent
  • hideshow 12 child comments
  • [–] 4 points 2 months ago (9 children)
  • [–] -1 points 2 months ago (8 children)
  • [–] 4 points 2 months ago (7 children)

    Usually the ones that accelerate the destruction of our planet, perpetuate sexist and racist biases, pray opon the weaknesses of how our minds work, exploit the underprivileged, devalue humanity and artistry, are annoying and overhyped, I think you get the picture.

  • source
  • parent
  • hideshow 7 child comments
  • [–] -1 points 2 months ago* (last edited 2 months ago) (6 children)

    So which algorithm is that? If you're hinting at manipulating the populace because in fact they are easily programmed via social network manipulation. Yeah anything done maliciously is of course bad but I think that comes down to the use of the tools and not the intrinsic nature of the tools themselves. Algorithms are just math, it's the will of the individuals wielding them that is corrupt and that seems to be your actual chief complaint.

  • source
  • parent
  • hideshow 6 child comments
  • [–] 2 points 2 months ago (5 children)

    I did not say that. You, like everyone else that says "its just a tool and technology is neutral", should read more books about ethics, politics and sociology. Everything we do is political. Nothing is "neutral".

  • source
  • parent
  • hideshow 5 child comments
  • [–] 0 points 2 months ago* (4 children)

    I'm not saying the use cases that some have chosen for certain tools is neutral. Their behavior is indeed bad and should be called out. Ai for instance is at it's core just a mathematical structure that has no intrinsic behavior, same for the concept of algorithms for social media or algorithms for drones or algorithms for the internet. Yeah some people use the internet to cause a lot of harm or even guide the outcomes of nations but is it the fault of html, CSS, and JavaScript? Not really. The problem is some companies, some people, some government officials and their choices.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 2 points 2 months ago (3 children)

    Firstly "AI" is a meaningless marketing term. And LLMs are fundamentally flawed. And especially anything by OpenAI/Anthropic/Meta/xAI should be avoided as they have shown zero regard for the actual negative impact of their products.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 0 points 2 months ago* (2 children)

    Sure, you know the term existed long before it became a financial buzz word right? Anyways in which sense are LLMs or NNs fundamentally flawed? I still believe your chief complaint is about the actions of people and not some mathematical structure much in the same way arguments about engagement algorithms on social media platforms is more about people and their choices than C++ or a binary tree.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 2 points 2 months ago (1 child)

    You know that it was a financial buzzword from the very beginning? I'll leave you with a reading suggestion: 'The AI con" from Timnit Gebru and Emily Bender. The latter also has various lectutes online about the topic.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 0 points 2 months ago* (last edited 2 months ago)

    How would you prefer people talk technology that allows machines to think, learn, and perform tasks that usually require human intelligence?

    We could encode some behavior with mechanical automaton but we can encode far more behavior and more complex behavior with electronic computers. There's no known strict limit beyond computation to the complexity of behavior we can encode. Now with a fully digitized fruit fly brain one can theorize about a fully digitized human brain. What would we call this if not artificial intelligence? How about other models that encode approximate behavior to human level intelligence?

    The term has linguistic meaning and isn't just marketing nonsense like calling food products all natural. That said there's definitely been a ramp up of people calling essentially if-else chains artificial intelligence, lots of abuse of the term.

  • source
  • parent
  • [–] -2 points 2 months ago (1 child)

    Pmuch this.

    How do they feel about machine language translations? It's been "AI" (ML model) since the late 90's.

    Traffic prediction?

    Medical scan data processing?

    And so so many more specific uses of ML models that have been a part of daily life for literal decades.

    Call out LLMs specifically.

  • source
  • parent
  • hideshow 1 child comment
  • [–] -2 points 2 months ago (3 children)

    not really related to the post, but NGL, it kinda fascinates me, how with the advent of LLMs, they became the ultimate punching bag for whenever something works bad, as if people didn't write even more horrible things without them (my regards to javascript and python).

  • source
  • hideshow 3 child comments
  • [–] -1 points 2 months ago (2 children)

    Will share the down votes with you, because I think the same.

    Claude opus is better than a significant amount of people I've worked with, and it's still significantly cheaper when now. Mind, we are not professional developers, but we need to develop a lot a DS solutions.

  • source
  • parent
  • hideshow 2 child comments
  • [–] -2 points 2 months ago

    What even is a "professional developer"? I mean, given that you aren't an egg-headed RnD developing new technology for someone like Nvidia, the job of a developer is 95% mind-numbing routine and typical tasks solved by applying ready-made patterns.

  • source
  • parent
  • [–] -2 points 2 months ago

    AI or not to AI is the same stories I heard from old time machinists when cnc productivity came for that industry.

  • source
  • 3 comments in c/cybersecuritymemes@lemmy.world view that post
    [–] 13 points 2 months ago (2 children)