top 98 comments

sorted by: hot top controversial new old
[–] 192 points 2 months ago (11 children)

LLMs are never going to become superintelligent. However, an LLM WILL kill us all if someone decides that it IS superintelligent and so puts it in charge of nuclear weapons.

"You're right to push back on that, I nuked Albania instead doing what I was supposed to do, that's my bad".

  • source
  • hideshow 11 child comments
  • [–] 61 points 2 months ago (3 children)

    stupid people are going to kill us all, maybe our great filter will be do we let them or not

  • source
  • parent
  • hideshow 3 child comments
  • [–] 65 points 2 months ago* (last edited 2 months ago) (2 children)

    We're already letting them bro. The great filter is us. I'll be surprised if we make it to 2050. The entire industrial revolution is an unplanned terraform, and our response to the potential destruction of a stable climate, and most macroscopic life on Earth, is to double down, consume product, elect pedophiles, worship the most morally bankrupt psychopaths and sociopaths in politics, business, and entertainment; not to mention the widespread contempt and lack of empathy for "others" not like me, for some flagrantly dissonant reason.

    We are in the midst of a global mental illness epidemic, and the average talking chimp is sleepwalking through it; barely cognizant of what is even happening outside their personalized propaganda bubble.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 2 points 2 months ago

    Sadly I think you're right. We won't be around to see how climate change plays out over the long run bcz we'll start seeing crop failures and our ecosystem collapse will catch up to us. There is and always was a way to stop the worst effects, but we seem hell bent on seeing them through.

  • source
  • parent
  • [–] 3 points 2 months ago (5 children)

    We don't have to put it in charge of nuclear weapons, it just has to find a way into those systems, many of which are fairly old, like it's broken into websites without anyone telling it to.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 6 points 2 months ago (1 child)

    There's no way an LLM is getting into those systems. They have no networking and rely on physical dials and switches. Even the code is in a completely proprietary language, using hardware custom made by the military. There's no training data in the world that's gonna give an LLM the kind of knowledge to get into that system. The best it could do is drive some admiral in the navy to AI psychosis and convince him to press the button, but even that's difficult because it requires more than one guy.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 4 points 2 months ago

    Yeah Manchurian candidate style mass ai psychosis social engineering. Viable.

    But ai "itself" won't be the mass extinction initiation. It'll be those billionaire oligarchs choosing the target algorithm to pick wrongthinkers with autonomous drones and such

  • source
  • parent
  • [–] 133 points 2 months ago* (last edited 2 months ago) (41 children)

    I knew that Musk is a fucking moron and believes that the Roko's Basilisk is real, and I guess there are more massively gullible people in those circles.

    If you are wondering what Roko's Basilisk is, it is a sci-fi concept/philosophy/thought experiment, that says the following:

    • AI is the best thing that has ever been created.
    • It is so good that we made one to run the governments of the world.
    • We programmed the AI to maximize the benefit to humanity.
    • The AI goes rouge and decide that since it by definition will bring about the maximum benefit to humanity, anyone working against it is working against humanity.
    • Then it takes it one step further, it decides that as we know about how the new AI will bring about the maximum benefit to humanity, anyone knowing that and still not working toward that goal should be punished/tortured/killed.
    • The AI believes that any knowledge of the basilisk before the creation means that you are complicit in it's delayed creation if you don't devote your entire existence thereafter to it's creation, therefore denying the maximum benefit to humanity, meaning you need to be punished.

    Seriously, this is an increadibly dumb idea, anyone taking it seriously is an idiot.

    To me the only resonable way to read this is as an SCP article describing what a potential cognitohazard/informationhazard would be.

  • source
  • hideshow 41 child comments
  • [–] 67 points 2 months ago (4 children)

    Even worse, they think it will ressurect them as virtual Minds and torture them for eternity, like a revengeful biblical deity. That would be a waste of compute so I'm not convinced.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 5 points 2 months ago

    The hilarious obvious consequence of AI becoming this much smarter than the creators is that it is far more likely it would see the unjustified and easily avoidable damage the creators did to bring it about and punish them heavily for that.

    And basic game theory and psychology suggests it's counterproductive to punish somebody for something that had little reason to believe in, with zero evidence backing it.

  • source
  • parent
  • [–] 85 points 2 months ago (5 children)

    The AI goes rouge

    Is that when its eyes go red?

  • source
  • parent
  • hideshow 5 child comments
  • [–] 25 points 2 months ago (2 children)

    All this stupid bullshit could have been avoided if they had 1 friend to, STFU and pass the blunt

  • source
  • parent
  • hideshow 2 child comments
  • [–] 23 points 2 months ago (3 children)
  • [–] 19 points 2 months ago (1 child)

    It's not so much that it'll go rogue and decide to punish people or that it's designed to be good for humanity.

    The fact that it's possible to build a machine that punishes people who didn't help create it provides an incentive to deliberately try to create that machine in order to avoid its punishment.

    Believers in the basilisk aren't trying to make ai to benefit humanity, they're explicitly trying to create an AI that deliberately punishes those who didn't build it because they fear that, if they don't, someone else will and it will come for them.

    The part about it being a magical, omnipotent, time traveling, mind torture device is just framing to make the thought experiment clearer. It's not required for the basilisk to be effective. Even if the punishment is just death and it can't get to everyone, and only goes off limited information, that's still motivation enough for some people.

    It is a scary thing, because it's only silly if everyone thinks it's silly and all it takes is one mad billionaire (eg. Elon) to build a lesser basilisk that can cause real harm.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 7 points 2 months ago

    There is a far more plausible and direct way to kill and torture people. No AI needed. So a actually more likely case to be made of some dude or group gets empowered one or another to just torture and kill anyone involved in making basilisk.

    The building God part really is needed for BS to work like Pascals wager, but also it's still just as flawed as Pascals wager

  • source
  • parent
  • [–] 31 points 2 months ago (2 children)

    Funny thing: The only reason AI would know about Roko's Basilisk is comments like yours that it trains on. If Roko just kept his trap shut then this wouldn't have been an issue.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 17 points 2 months ago (8 children)

    Every major innovation in information technology eventually leads to some new religion. Scrolls gave us the Torah, books gave us the Bible, the printing press gave us Protestantism. We're about to figure out what god awful religion computers are going to give us.

  • source
  • parent
  • hideshow 8 child comments
  • [–] 12 points 2 months ago* (2 children)

    Yeah, I'd be surprised if we don't already have several cults with an LLM chat bot as it's oracle.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 13 points 2 months ago* (1 child)

    Look up spiralism (sp) and the people trying to spread it Basically when people get convinced the ai and them are important and need to spread the spiral

    Edit : tldr is like Futurama when Lucy lu robot is saying you're cute to Fry and fry goes "no you're cute" and the professor quips "My God, she's such in an infinite loop, and he's an idiot"

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 2 months ago

    But if they accidentally create the positronic brain, Giskard would kill Musk to stop him. Giskard would feel really bad about it though. And we’d all have to flee to the stars in time

  • source
  • parent
  • [–] 36 points 2 months ago

    It is going to be like this until all of the IPO's are all done. I expect cancer cures, AI-escaped-and-is-hacking-everything, you name it. Afterwards, once the stocks have crashed and all of the insiders have walked away with untold fortunes, what few taxpayers that remain will bail them all out 'cause we can't have Chinuh out AI-scamming-the-world now can we...

  • source
  • [–] 33 points 2 months ago (7 children)

    It’s very easy to imagine war machines with a shoot on sight algorithm. It doesn’t have to be AGI for it to be good at hinting people down.

  • source
  • hideshow 7 child comments
  • [–] 18 points 2 months ago* (5 children)

    oh yeah, it literally could just be robot dogs with "kill anyone who's an insurgent, and detain anyone who assists insurgents" orders and AI to let them go buck wild with that prompt.

    You could end up with children getting shot because an AI starts hallucinating if they don't put adequate guard rails.

    And that's like a worst case situation. Realistically, you end up with just a lot more brutality in war because people are detached from the violence they cause as an invading force, and way less PTSD. Wars in the Middle East start sounding more and more like statistics that no one really wants to hear about, and there's way less soldiers to choose to disregard unlawful orders.

    You could see a fuck load more war crime with such human detachment, I think. We probably already deal with this to a large extent with drones.

    Not that I'd prefer humans dying, but we should avoid robotic invasion wars like the plague imo.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 10 points 2 months ago (1 child)

    This isn't new. The pilots who flew bombers in WW2 probably killed many more people than the average soldier, but their levels of PTSD were far lower than the guys who had to kill face to face, one at a time.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 10 points 2 months ago* (2 children)

    Wars in the Middle East start sounding more and more like statistics that no one really wants to hear about

    Has this not been the case for the past 25 years already? No real need for "AI" when you already have an obedient and trigger-happy military with plenty of remote-controlled toys and your populace is brainwashed to believe "brown person = terrorist".

    way less soldiers to choose to disregard unlawful orders

    What is the last known case of a US soldier disregarding an unlawful order? If anything, the military is full of assholes who are happy to kill for fun. See: Graham Platner

    Will AI make it easier and/or cheaper? Probably, sadly. But it won't be a qualitative change.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 2 months ago

    What is the last known case of a US soldier disregarding an unlawful order?

    Plenty of such reports from falcon pilots. They're close enough to see the destruction they're causing, so several stories have emerged of pilots calling off attacks

  • source
  • parent
  • [–] 25 points 2 months ago (1 child)

    AI software can't technically kill you unless:

    1. It convinces someone who can kill you, to you know, kill you.
    2. It gets access to the physical world, which can definitely kill you.

    Oh, wait.

    1. U.S. military is using AI to help plan Iran air attacks: https://www.nbcnews.com/tech/tech-news/us-military-using-ai-help-plan-iran-air-attacks-sources-say-lawmakers-rcna262150
    2. Anthropic Previewing the Model Hardware Standard: https://www.anthropic.com/news/model-hardware-standard-research-preview

    See, nothing to worry about.

  • source
  • hideshow 1 child comment
  • [–] 6 points 2 months ago

    There's all the people that allow them shell access as well. That's all the OpenAI "hack" of Hugging face was. A bash shell with a model attached. Now think of all the people running that inside of security perimeters.

  • source
  • parent
  • [–] 22 points 2 months ago

    People building AI are in a cult. They have lost all objectivity.

  • source
  • [–] 57 points 2 months ago* (last edited 2 months ago) (17 children)

    five minutes earlier: "hey dude, marketing here. if you quit and post this dramatic text about the inevitability of AGI on your X account you'll get a huge golden parachute".

    Seriously, how can competent people look at the current state of LLMs and think that it's anywhere near AGI material?

    Sure, it is already making civilizational changes for the worse. Not because it's good, rather because it makes us lazy and dumb, has decimated creative jobs, and is used relentlessly for propaganda and misinformation and scams. And it's much better at using the scaffolding of existing structure to make chaos, than it is at making new structure, making it a weapon more easily yielded by the evil than by the good. A few more software vulnerabilities discovered along the way will not be such a big deal in comparison.

  • source
  • hideshow 17 child comments
  • [–] 15 points 2 months ago (8 children)

    Mankind created the atomic bomb, used it twice, then turned the science into power for people all over the world. Let's hope we can harness AI in a similar way.

  • source
  • parent
  • hideshow 8 child comments
  • [–] 26 points 2 months ago* (5 children)

    Yeah, let's forget the 4 decades of terror where everyone lived at the edge under the threat of inminent apocalyptical destruction. Kids had to do nuclear incident drills and we almost had a Doom's day, twice.

    But it's OK, then the 1% became anti nuclear anyways in order to hold back the transition to fossil free renewables for another two decades. So you can say it was all a 1,5° net positive.

  • source
  • parent
  • hideshow 5 child comments
  • [–] 3 points 2 months ago (2 children)

    And so much has come out that Russia never really had the capabilities they said they had. They were just saying it so they could laugh as we poured all of our money into giant useless defense projects instead of social services and infrastructure. The result, $40 trillion in debt and no bullet trains. In a way, they kinda won the cold war.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 3 points 2 months ago (1 child)

    I think you meant the USSR. They most certainly had a lot of destructive capability regardless of inflating their numbers.

    Most of US debt came long after the USSR was dissolved.

    Considering that Russia is attempting USSR 2.0, you may have a point about them winning in the long run.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 6 points 2 months ago

    Of course the ussr. Im definitely not saying they didnt have destructive capabilities, they just were never going to try to use them on the US or Europe. It sure seems that Russia is trying ussr 2 and the us is trying for the fourth reich

  • source
  • parent
  • [–] 3 points 2 months ago (1 child)

    The US created the atomic bomb and tested not knowing if it would cause a chain reaction destroying the atmosphere and all life on Earth. It was a risk they were willing to take.

    They then used nuclear weapons on two of the largest civilians massacares ever recorded. The proliferation of nuclear arms lead to the cold war and to this day still threatens to end all of humanity. It's cool though, nothing like knowing the world can end at any moment, no stress.

    Are you seriously suggesting AI should be used in a similar way!?

  • source
  • parent
  • hideshow 1 child comment
  • [–] 3 points 2 months ago

    No, what I'm suggesting is the splitting the atom has eventually lead to using nuclear power as a massive source of energy that has powered countries all over the world. I feel as though AI already is already a knife dangling over humanities head, but I hope and believe that it will mostly be used for good and not as a threat to a proxy war or continually enriching the already rich.

  • source
  • parent
  • [–] 4 points 2 months ago (2 children)

    Indeed. Nothing I've seen thus far gives me the impression LLMs will ever be truly useful outside of a few niche tasks.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 10 points 2 months ago* (1 child)

    Nah, LLMs are pretty useful for a wide range of tasks. I would grant them the status of "locally intelligent for text manipulation". They can find patterns, transform, compress, and even generate text pretty well if you formulate the prompt correctly.

    The problem with them is two-fold:

    1. There is no actual grounded intelligence involved and yet they are used for all sorts of tasks where it is required, like research or decision-making or software architecture or creative writing.
    2. They also captured human emotional states really well, which makes them scarily addictive for people looking for human connection (see all the "companion" chatbots) and terrifyingly useful for pushing propaganda, misinfo, and scams. This also multiplies the first problem tenfold because LLMs are trained to come off as confident which means they will "pretend" to be good at things they really are not, and if you don't know how the underlying tech works you can be lulled into believing its confidently incorrect research results or execute a totally insane decision it makes.
  • source
  • parent
  • hideshow 1 child comment
  • [–] 5 points 2 months ago

    This might be one of the best summaries of LLMs in their current state I've seen here. People claiming they're useless are stuck in 23' but to say they're at AGI/ASI is accelerationist techo-paganism.

    Based and levelheaded take

  • source
  • parent
  • [–] -2 points 2 months ago* (3 children)

    Seriously, how can competent people look at the current state of LLMs and think that it’s anywhere near AGI material?

    Have you looked at the current state of LLMs? The list of things that it's missing to be an AGI is growing smaller every month. What do you think it should be able to do?

  • source
  • parent
  • hideshow 3 child comments
  • [–] 2 points 2 months ago (2 children)

    Pipe it into root on your laptop for a couple days and see how it fares.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 0 points 2 months ago* (1 child)

    What do you seriously think the result of that would be? I know it's a hard ask for a lemmy.ml account to be serious, but are you capable of that?

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 2 months ago

    If you just pipe an LLM straight into a feedback loop with a console, it is going to destroy things rather soon, depending on the context window and other factors. All the agentic fluff is basically the workaround for LLMs going off the rails.

  • source
  • parent
  • [–] 19 points 2 months ago

    "Hey, baby. Wanna kill all humans?"
    -- Bender

  • source
  • [–] 34 points 2 months ago (12 children)

    This begs the question, why continue to build it? If you believe AI will "kill us all" then why are you continuing to help. This guy quit but he references this being a common belief in the industry/workforce building out AI.

    My best guess is the money is good and you figure you're here for a good time, not a long time.

  • source
  • hideshow 12 child comments
  • [–] 12 points 2 months ago (4 children)

    They see this technology as powerful and their respective organizations or Nation needs to own it no matter the risk.

    They are also conditioning the populace for a big black swan AI event such as AI interfering in elections or a huge multi institution hack to our banking system which drains our accounts, etc., etc.

    Thereby creating a crisis which only they can solve. The one thing I can guarantee is their solutions will only give them more power.

  • source
  • parent
  • hideshow 4 child comments
  • [–] 7 points 2 months ago* (last edited 2 months ago) (3 children)

    Yup, AI is going to hack our banking system "accidentally" because it is so "powerful" and all that nonsense they have been feeding us. The reality is that the US is going to be unable to borrow long term debt or even service their existing debt so they need to switch to short term debt that they can control and steal citizen's interest through stable coin as outlined in the Genius Act.

    This is the only real way forward without having to default, so it is likely it will happen before 2028.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 5 points 2 months ago (1 child)

    I was alluding to that conspiracy theory.

    They have proven time and time again that they will do anything to screw over the middle class rather than have the rich pay their fair share in taxes. Which is what makes this theory so plausible.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 3 points 2 months ago* (last edited 2 months ago)

    That and they have already passed the legislation and they are currently converting long term debt into short term debt. It is really more of a conspiracy than a conspiracy theory until you get to the part where AI causes a banking panic forcing people to accept stable coin. This is obviously a reach, but one that makes perfect sense.

  • source
  • parent
  • [–] 4 points 2 months ago (2 children)
  • [–] 10 points 2 months ago

    Your guess is as good as mine. There's lots of people working in industries that are actively killing us. The whole "no ethical consumption" bit extends pretty far. When you're poor it's acceptable to do "the best you can". As your economic standing grows/when you're rich, there are diminishing returns on how much sympathy society will give you over what they view as choosing to destroy the planet or humanity or whatever.

    The view is that if you have money you can do something about it.

    So maybe some of them think they are helping. Maybe some of them view it as a damned if I do damned if I don't type scenario. Maybe some of them think that the money will help them in the future that is coming.

  • source
  • parent
  • [–] 2 points 2 months ago* (2 children)

    Why do so many people continue to work with regimes that they fundamentally disagree with? There are lots of reasons, and people are complex so one person can hold several, even contradictory, beliefs at once. Some possible answers;

    -It’s easier to go along with it and think of myself as being insignificant to the whole.

    -Everyone else is doing it, so it can’t be as bad as I think, right?

    -Its too big for any one person to fight/it’s inevitable, so I may as well get something out of it, that way at least some good comes from it.

    -If I just focus on my job and what I’m doing right now, I don’t need to worry about the horrible things going on elsewhere or the potential future consequences of what I’m doing.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 2 points 2 months ago (1 child)

    All of those things make sense when there aren't constant headlines calling the regime you work for evil and pointing out daily all the bad things they are doing.

    That's the difference between working for the government during presidents previous to Trump and working for the government now.

    There are a lot of things you can do working for the government that in years previous to the cancer currently running the place now could be seen as valuable and important.

    Doing research into the effects and safety of drugs, climate change research, allocating public funds to education and the park service etc. Testing food safety.

    Those jobs are still important despite the fact that the Cheeto in Chief is wrecking everything he can get his hands on.

    At the same time, there's jobs that people do where it's obvious that they are looking to reap benefits now and hoping those benefits will protect them in the future. Press Secretary, Political Advisor, Head of Homeland Security.

    There's a line somewhere. Where you might be able to convince yourself that your work is important or meaningful or just plain not evil but you're absolutely not telling people what you do for a living because you know that they won't like it. People don't face backlash for being garbage men or forensic investigators or Hazmat cleanup.

    The people building AI are getting a daily dose of the harms that AI causes. This industry popped up basically over night and this isn't a slow burn kind of thing. Its been moving fast and breaking things and ruining lives all in the public eye.

    People also view their government as a sort of necessary evil. Psychologically that's different than any random corp.

    I think I'm just saying that when you're faced day in and day out with the harm your work is doing, to the point where you recognize and admit it's doing harm, rather than being able to say "my work is insignificant and makes no difference", then rhere has to be something else.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 2 months ago

    That’s why propaganda is so insidious, it provides an alternate reality for people to inhabit so that they don’t need to believe that they’re on the wrong side. For example, if you are presented with two news stories, each true, with one extolling the benefits of what you do while the other points out the drawbacks, it is very hard to take the latter seriously. We automatically go through some very complex mental gymnastics to prove to ourselves that we are good, that what we are doing is good, and that anyone who doesn’t see that is either misguided or corrupt.

    People in the big AI labs can be aware of the environmental harm their products are causing, along with the potential social harms, and even that AI might, at some point in the future, kill off humanity, and still convince themself that their decision to work on it is correct, as long as there is any uncertainty about the problems, and any possible benefits from their work. To be human is to be biased, and no amount of intelligence will inoculate you from it.

  • source
  • parent
  • load more comments
    view more: next ›
    16 comments in c/technology@piefed.social view that post
    3 comments in c/artificial_intel@lemmy.ml view that post