1123
Pull the plug already (lemmy.world)
submitted 3 weeks ago by to c/fuck_ai@lemmy.world
210 comments fedilink hide all child comments
 
you are viewing a single comment's thread
view the rest of the comments
[–] 0 points 3 weeks ago (5 children)

You're explaining why it went rogue, not that it didn't go rogue.

Per the dictionary going rogue means:

to begin to behave in an independent or uncontrolled way that is not authorized, normal, or expected

So the agents that hacked hugging face acted independently without authorization as shown by the prompts which did not tell them to hack or do anything malicious. You might say this is normal but none of the other trials openai was doing showed this behavior, it was only for the impossible trial. You might say it's expected but if you were able to determine that the prompt they gave it would result in hugging face getting hacked then you must be high af on the melange spice.

If a general goes rogue and starts bombing civilians, you could explain it by saying that he read machiavelli or mein kampf and thought this was the best way to achieve his goals, but that doesn't mean the general didn't go rogue. You could say that this was expected because there were "obvious signs" that people missed but you're working on hindsight, it's easy to say something should've been expected when it's already happened.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 5 child comments
    •
  • [–] 2 points 3 weeks ago* (last edited 3 weeks ago) (3 children)

    to begin to behave in an independent or uncontrolled way that is not authorized, normal, or expected

    This definition is full of holes. Your microwave isn't going rogue when it burns your food. Your car isn't going rogue when you blow a tire.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 0 points 3 weeks ago* (last edited 3 weeks ago) (2 children)

    No, because that is expected authorized normal behavior. You gave the command / authorized the microwave to run for 2 minutes and it ran for 2 minutes. The effect of the food getting hot or even burning is expected behavior.

    The agents that hacked hugging face were given the command to use a specific exploit to get a secret in a sandbox. After they discovered it was impossible it broke out of its sandbox and hacked an unrelated service to try and trick the judge. That is not expected authorized behavior.

    If you tell someone to bake some dough for 30 minutes and that person does it and it burns that's not acting rogue. If that person discovers the oven is broken so they break into your neighbors house to plant some bread to claim they completed the task, that person went rogue.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 3 weeks ago* (last edited 3 weeks ago)

    The popcorn button burning your popcorn doesn't seem like it'd be authorized or expected. I guess it is normal, though.

    You skipped over the second half of my comment. Your car blowing a tire unexpectedly on the road and causing you to swerve into a ditch against your will is not expected or authorized or normal behavior. It is, in fact, the opposite of all those things!

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 3 weeks ago* (last edited 3 weeks ago)

    An LLM “agent” is not a person. It's software. It has no agency, no intentions, no responsibility.

    It could malfunction, you can operate it poorly and irresponsibly, but it can't “go rogue”.

    When windows update fubars your system to the point that you have to reinstall it or switch to a proper OS, it's not “going rogue”, it's just being the buggy vibecoded mess that it is, and at this point that's just an expectable outcome of using it.

    An LLM couldn't care less for the intention of your prompt if it was actually capable of caring about anything.

    It's just a piece of software that will add words after your prompt in a manner statistically coherent with its training model. Nothing more, nothing less.

    It doesn't know what your prompt means, or what the words it's producing mean. It's not capable of knowing anything.

    You can plug its output and input to a command prompt and call it an agent, but that's not giving it any actual agency, that's just you setting a random time on the microwave without looking, putting your fingers in your ears, and going “la la la, wasn't me ♫” while claiming to have no responsibility over the results (unless the food turns out good, in which case you probably will claim all the responsibility).

    If you don't expect the autocomplete machine trained on internet garbage to produce the kind of garbage it was trained on that's not the machine “going rogue”, that's you being unqualified to use it, and using it for a purpose it's inadequate for.

    The only one going rogue (and capable of going rogue) here, as with any other software, is the user.

  • permalink
  • fedilink
  • source
  • parent
  • [–] 1 point 3 weeks ago

    My car isn't going rogue if I drive it into a tree. Or if I leave the hand brake off and it rolls off a cliff.

    LLMs can't act independently. They autocomplete text, that's all they do.

    Sure, you can feed the output of one into itself and use the output of some other software as a prompt and call it an agent in a futile attempt to imagine it has some sort of agency and try to exempt yourself of responsibility, but that's just you trying to rationalize driving the car into the tree, possibly due to psychosis and brainrot caused by LLM exposure, or in OpenAI's case as a criminal marketing stunt.

    Can I predict what specific mayhem my poorly prompted agent (from which I intentionally removed any safeguards) in my (probably intentionally) incompetently secured sandbox will cause?

    Not specifically, no. But I can be sure that if I try enough prompt variations it will end up producing the kind of petty evil bullshit an internet troll or a villain in a thriller would produce, because that's a statistically significant part of its training data, and then use the ensuing mayhem as a publicity stunt to both show potential investors (and deluded brainrotten users) how powerful my so called thinking machine is, and spew FUD into politicians' ears to try and get them to ban foreign competitors and open weight models, which is what OpenAI did.

  • permalink
  • fedilink
  • source
  • parent