▲ 271 ▼ Signal president warns AI agents are making encryption irrelevant (cyberinsider.com) submitted 9 months ago by nemeski@piefed.social to c/cybersecurity@sh.itjust.works 46 comments fedilink hide all child comments
[–] TribblesBestFriend@startrek.website 10 points 9 months ago (13 children) Some one could TL; DR me ? permalink fedilink source hideshow 13 child comments replies: [–] Voroxpete@sh.itjust.works 61 points 9 months ago* (last edited 9 months ago) (4 children) He's She's talking specifically about the idea of embedding AI agents in operating systems, and allowing them to interact with the OS on the user's behalf. So if you think about something like Signal, the point is that as it leaves your device the message is encrypted, and only gets decrypted when it arrives on the device of the intended recipient. This should shut down most "Man in the middle" type of attacks. It's like writing your letters in code so that if the FBI opens them, they can't read any of it. But when you add an AI agent in the OS, that's like dictating your letter to an FBI agent, and then encrypting it. Kind of makes the encryption part pointless. permalink fedilink source parent hideshow 4 child comments replies: [–] TribblesBestFriend@startrek.website 11 points 9 months ago Thanks ! permalink fedilink source parent [–] eleijeep@piefed.social 7 points 9 months ago (1 child) He’s talking specifically She* permalink fedilink source parent hideshow 1 child comment replies: [–] Voroxpete@sh.itjust.works 4 points 9 months ago My bad. Thanks for the correction. permalink fedilink source parent [–] MeThisGuy@feddit.nl 6 points 9 months ago like using Gboard? permalink fedilink source parent [–] French75@slrpnk.net 22 points 9 months ago* (last edited 9 months ago) (1 child) Encrypted apps like Signal encrypt messages in a way that only you and the recipient can decrypt and read. Not even Signal can decrypt them. However it has always been the case that another person could look over your shoulder and read the messages you send, who you're sending them to, and so on. Pretty obvious, right? What the author and Signal are calling out here is that all major commercial OSes are now building in features that "look over your shoulder." But it's worse than that because they also record every other device sensor's data. Windows Recall is the easiest to understand. It is a tool build into windows (and enabled by default) that takes a screenshot a few times per second. This effectively capture a stream of everything you do while using windows; what you browse, who you chat with, the pron you watch, the games you play, where you travel, and who you travel with or near. If you use "private" message tools like Signal, they'll be able to see who you are messaging and read the conversations, just as if they were looking over your shoulder, permanently. They claim that for an AI agent to serve you well, it needs to know everything it can about you. They also make dubious claims that they'll never use any of this against you, but they also acknowledge that they comply with court orders and government requests (to varying degrees). So... if you trust all of these companies and every government in the world, there's nothing to worry about. permalink fedilink source parent hideshow 1 child comment replies: [–] TribblesBestFriend@startrek.website 2 points 9 months ago Thanks permalink fedilink source parent [–] trevor@lemmy.blahaj.zone 18 points 9 months ago (1 child) "Agentic" LLMs are turning garbage operating systems, like Microslop Winblows, into hostile and untrusted environments where applications need to run. A primary example given is how Recall constantly captures your screen and turns the image data into text that can be processed by Microslop, thus making the fact that Signal is end-to-end encrypted largely irrelevant, since your OS is literally shoulder-surfing you at all times. This is made worse by the fact that the only workaround that application developers can use to defend against this surveillance is to implement OS DRM APIs, which are also controlled by the hostile entity. permalink fedilink source parent hideshow 1 child comment replies: [–] TribblesBestFriend@startrek.website 4 points 9 months ago Thanks ! permalink fedilink source parent [–] UnspecificGravity@piefed.social 10 points 9 months ago* (1 child) Your operating system and half the software you use has integrated spyware that can read anything you see on your computer or phone as free text and use that information to notify state actors or just whoever the fuck they want of the contents. It doesn't matter that the message was encrypted between you and the other person when they can spy directly on your device. Its like passing a coded note to a friend in class and then they open it and just read it out loud to everyone sitting there. Didn't really matter that you encoded it. permalink fedilink source parent hideshow 1 child comment replies: [–] TribblesBestFriend@startrek.website 1 point 9 months ago Thank ! permalink fedilink source parent [–] kingofras@lemmy.world 15 points 9 months ago (1 child) During the interview, she described how AI agents are marketed as helpful assistants but require sweeping permissions to work. As Whittaker explained, these systems are pitched as tools that can coordinate events or communicate on a user’s behalf, but to do so they must access calendars, browsers, payment methods, and private messaging apps like Signal, placing decrypted messages directly within reach of the operating system permalink fedilink source parent hideshow 1 child comment replies: [–] TribblesBestFriend@startrek.website 3 points 9 months ago Thanks! permalink fedilink source parent
[–] Voroxpete@sh.itjust.works 61 points 9 months ago* (last edited 9 months ago) (4 children) He's She's talking specifically about the idea of embedding AI agents in operating systems, and allowing them to interact with the OS on the user's behalf. So if you think about something like Signal, the point is that as it leaves your device the message is encrypted, and only gets decrypted when it arrives on the device of the intended recipient. This should shut down most "Man in the middle" type of attacks. It's like writing your letters in code so that if the FBI opens them, they can't read any of it. But when you add an AI agent in the OS, that's like dictating your letter to an FBI agent, and then encrypting it. Kind of makes the encryption part pointless. permalink fedilink source parent hideshow 4 child comments replies: [–] TribblesBestFriend@startrek.website 11 points 9 months ago Thanks ! permalink fedilink source parent [–] eleijeep@piefed.social 7 points 9 months ago (1 child) He’s talking specifically She* permalink fedilink source parent hideshow 1 child comment replies: [–] Voroxpete@sh.itjust.works 4 points 9 months ago My bad. Thanks for the correction. permalink fedilink source parent [–] MeThisGuy@feddit.nl 6 points 9 months ago like using Gboard? permalink fedilink source parent
[–] TribblesBestFriend@startrek.website 11 points 9 months ago Thanks ! permalink fedilink source parent
[–] eleijeep@piefed.social 7 points 9 months ago (1 child) He’s talking specifically She* permalink fedilink source parent hideshow 1 child comment replies: [–] Voroxpete@sh.itjust.works 4 points 9 months ago My bad. Thanks for the correction. permalink fedilink source parent
[–] Voroxpete@sh.itjust.works 4 points 9 months ago My bad. Thanks for the correction. permalink fedilink source parent
[–] French75@slrpnk.net 22 points 9 months ago* (last edited 9 months ago) (1 child) Encrypted apps like Signal encrypt messages in a way that only you and the recipient can decrypt and read. Not even Signal can decrypt them. However it has always been the case that another person could look over your shoulder and read the messages you send, who you're sending them to, and so on. Pretty obvious, right? What the author and Signal are calling out here is that all major commercial OSes are now building in features that "look over your shoulder." But it's worse than that because they also record every other device sensor's data. Windows Recall is the easiest to understand. It is a tool build into windows (and enabled by default) that takes a screenshot a few times per second. This effectively capture a stream of everything you do while using windows; what you browse, who you chat with, the pron you watch, the games you play, where you travel, and who you travel with or near. If you use "private" message tools like Signal, they'll be able to see who you are messaging and read the conversations, just as if they were looking over your shoulder, permanently. They claim that for an AI agent to serve you well, it needs to know everything it can about you. They also make dubious claims that they'll never use any of this against you, but they also acknowledge that they comply with court orders and government requests (to varying degrees). So... if you trust all of these companies and every government in the world, there's nothing to worry about. permalink fedilink source parent hideshow 1 child comment replies: [–] TribblesBestFriend@startrek.website 2 points 9 months ago Thanks permalink fedilink source parent
[–] TribblesBestFriend@startrek.website 2 points 9 months ago Thanks permalink fedilink source parent
[–] trevor@lemmy.blahaj.zone 18 points 9 months ago (1 child) "Agentic" LLMs are turning garbage operating systems, like Microslop Winblows, into hostile and untrusted environments where applications need to run. A primary example given is how Recall constantly captures your screen and turns the image data into text that can be processed by Microslop, thus making the fact that Signal is end-to-end encrypted largely irrelevant, since your OS is literally shoulder-surfing you at all times. This is made worse by the fact that the only workaround that application developers can use to defend against this surveillance is to implement OS DRM APIs, which are also controlled by the hostile entity. permalink fedilink source parent hideshow 1 child comment replies: [–] TribblesBestFriend@startrek.website 4 points 9 months ago Thanks ! permalink fedilink source parent
[–] TribblesBestFriend@startrek.website 4 points 9 months ago Thanks ! permalink fedilink source parent
[–] UnspecificGravity@piefed.social 10 points 9 months ago* (1 child) Your operating system and half the software you use has integrated spyware that can read anything you see on your computer or phone as free text and use that information to notify state actors or just whoever the fuck they want of the contents. It doesn't matter that the message was encrypted between you and the other person when they can spy directly on your device. Its like passing a coded note to a friend in class and then they open it and just read it out loud to everyone sitting there. Didn't really matter that you encoded it. permalink fedilink source parent hideshow 1 child comment replies: [–] TribblesBestFriend@startrek.website 1 point 9 months ago Thank ! permalink fedilink source parent
[–] TribblesBestFriend@startrek.website 1 point 9 months ago Thank ! permalink fedilink source parent
[–] kingofras@lemmy.world 15 points 9 months ago (1 child) During the interview, she described how AI agents are marketed as helpful assistants but require sweeping permissions to work. As Whittaker explained, these systems are pitched as tools that can coordinate events or communicate on a user’s behalf, but to do so they must access calendars, browsers, payment methods, and private messaging apps like Signal, placing decrypted messages directly within reach of the operating system permalink fedilink source parent hideshow 1 child comment replies: [–] TribblesBestFriend@startrek.website 3 points 9 months ago Thanks! permalink fedilink source parent
[–] TribblesBestFriend@startrek.website 3 points 9 months ago Thanks! permalink fedilink source parent