Jen Easterly says most breaches stem from bad software, and smarter tech could finally clean it up Ex-CISA head Jen Easterly claims AI could spell the end of the cybersecurity industry, as the sloppy software and vulnerabilities that criminals rely on will be tracked down faster than ever.…

all 2 comments

sorted by: hot top controversial new old
[–] 6 points 1 year ago*

Gosh I love how nonsense like this gets buzzsawed by The Register's comment section.

I mean..

Ultimately, she said, "if we're able to build and deploy and govern these incredibly powerful technologies in a secure way, I believe it will lead to the end of cybersecurity."

This reads like an Onion article written by a cybersecurity expert.

  • source
  • Anyone still under the illusion the technocrats are a meritocracy?

  • source
  • 76 comments in c/programmer_humor@programming.dev view that post
    [–] 177 points 1 year ago (11 children)

    couldn't ai, then also, break code faster than we could fix it ?

  • source
  • hideshow 11 child comments
  • [–] 45 points 1 year ago (3 children)

    I mean, at a high level it is very much the concept of ICE from Gibson et al back in the day.

    Intrusion Countermeasures Electronics. The idea that you have code that is constantly changing and updating based upon external stimuli. A particularly talented hacker, or AI, can potentially bypass it but it is a very system/mental intensive process and the stronger the ICE, the stronger the tools need to be.

    In the context of AI on both sides? Higher quality models backed by big ass expensive rigs on one side should work for anything short of a state level actor... if your models are good (big ol' "if" that).

    Which then gets into the idea of Black ICE that is actively antagonistic towards those who are detected as attempting to bypass it. In the books it would fry brains. In the modern day it isn't overly dissimilar from how so many VPN controlled IPs are just outright blocked from services and there is always the risk of getting banned because your wifi coffee maker is part of a botnet.

    But it is also not hard to imagine a world where a counter-DDOS or hack is run. Or a message is sent to the guy in the basement of the datacenter to go unplug that rack and provide the contact information of whoever was using it.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 8 points 1 year ago (2 children)

    In the context of AI on both sides? Higher quality models backed by big ass expensive rigs on one side should work for anything short of a state level actor… if your models are good (big ol’ “if” that).

    Turns out Harlan Ellison was a goddamn prophet when he wrote I Have No Mouth And I Must Scream.

  • source
  • parent
  • hideshow 2 child comments
  • [–] 18 points 1 year ago (3 children)

    AI WRITES broken code. Exploiting is is even easier.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 2 points 1 year ago

    A few years back someone made virus that connected to an llm server and kept finding ways to infect computers in the simulated network. I think it was kind of successful. Not viable for a virus though, but an interesting idea non the less

  • source
  • parent
  • [–] 70 points 1 year ago (4 children)
  • [–] 57 points 1 year ago

    Execs and managers showing Dunning-Kruger in full effect.

  • source
  • [–] 42 points 1 year ago

    AI is opening so many security HOLES. Its not solving shit. AI browsers and MCP connectors are wild west security nightmares. And that's before you even trust any code these things write.

  • source
  • [–] 41 points 1 year ago (1 child)

    At this point, they're just rage baiting and saying random shit to squeeze that bubble before it bursts.

  • source
  • hideshow 1 child comment
  • [–] 33 points 1 year ago

    As usual, the biggest advocates for AI are the ones who understand its limitations the least.

  • source
  • [–] 33 points 1 year ago
    [–] 31 points 1 year ago* (2 children)

    I have worked as a pentester and eventually a Red Team lead before leaving foe gamedev, and oh god this is so horrifiying to read.

    The state of the industry was alredy extremely depressing, which is why I left. Even without all of this AI craze, the fact that I was able to get from a junior to Red Team Lead, in a corporation with hundreds of employees, in a span of 4 years is already fucked up, solely because Red Teaming was starting to be a buzz word, and I had passion for the field and for Shadowrun while also being good at presentations that customers liked.

    When I got into the team, the "inhouse custom malware" was a web server with a script that pools it for commands to run with cmd.exe. It had a pretty involved custom obfuscation, but it took me lile two engagements and the guy responsible for it to leave before I even (during my own research) found out that WinAPI is a thing, and that you actually should run stuff from memory and why. And I was just a junior at the time, and this "revelation" got me eventually a unofficial RT Lead position, with 2 MDs per month for learning and internal development, rest had to be on engagements.

    And even then, we were able to do kind of OK in engagements, because the customers didn't know and also didn't care. I was always able to come up with "lessons learned", and we always found out some glaring sec policy issues, even with limited tools, but the thing is - they still did not care. We reported something, and two years ago they still had the same bruteforcable kerberos tickets. It already felt like the industry is just a scam done for appearances, and if it's now just AIs talking to the AIs then, well, I don't think much would change.

    But it sucks. I love offensive security, it was really interresting few years of my carreer, but ot was so sad to do, if you wanted to do it well :(

  • source
  • hideshow 2 child comments
  • [–] 5 points 1 year ago (1 child)

    Seeing all these AI ideas, i think security is about to get hugely more important in the near future.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 8 points 1 year ago

    Definitely, but the issue is that even the security companies that actually do the assesments also seem to be heavily transitioning towards AI.

    To be fair, in some cases, ML is actually really good (i.e in EDRs. Bypassing a ML-trained EDR is really annoying, since you can't easily see what was it that triggered the detection, and that's good), and that will carry most of the prevention and compensate for the vulnerable and buggy software. A good EDR and WAF can stop a lot. That is, assuming you can afford such an EDR, AV won't do shit - but unless we get another Wannacry, no-one cares that a few dozen of people got hacked through random game/app, "it's probably their fault for installing random crap anyway".

    I've also already seen a lot of people either writing reports with, or building whole tools that run "agentic penetration tests". So, instead of a Nessus scan, or an actual Red Teamer building a scenario themselves, you get a LLM to write and decide a random course of action, and they just trust the results.

    Most of the cybersecurity SaaS corporates didn't care about the quality of the work before, just like the companies that are actually getting the services didn't care (but had to check a checkbox). There's not really an incentive for them to do so, worst case you get into a finger-pointing scenario ("We did have it pentested" -> "But our contract says that we can't 100% find everything, and this wasn't found because XYZ... Here's a report with our methodology that we did everything right"), or the modern equivalent of "It was the AI's fault", maybe get a slap on the wrist, but I think that it will not get more important, but way, way more depressing than it already was three years ago.

    I'd estimate it will take around a decade of unusable software and dozens of extremely major security breaches before any of the large corporations (on any side) concedes that AI was really, really stupid idea. And at that time they'll probably also realize that they can just get away with buggy vulnerable software and not care, since breaches will be pretty common place, and probably won't affect larger companies with good (and expensive) frontline mitigation tools.

  • source
  • parent
  • [–] 28 points 1 year ago (9 children)

    I tried using AI in my rust project and gave up on letting it write code. It does quite alright in python, but rust is still too niche for it. Imagine trying to write zig or Haskell, it would make a terrible mess of it.

    Security is an afterthought in 99.99% of code. AI barely has anything to learn from.

  • source
  • hideshow 9 child comments
  • [–] 32 points 1 year ago (3 children)

    If you're using Hannah Montana Linux you can just open a terminal and type "write me ____ in the language ____" and the Hannai Montanai will produce perfectly working code every time.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 4 points 1 year ago

    Even in Python you have to keep it siloed. You have to drip feed it pieces because if you give it the whole script it'll eat comments, straight up chop out pieces so you end up with something like

     def myFunction():
          # ...start of your function here...
    

    replacing actual code.

  • source
  • parent
  • [–] 2 points 1 year ago (2 children)

    Mitchell Hashimoto writes a lot of Zig with AI (and this interview is almost a year old), see: https://www.youtube.com/watch?v=YQnz7L6x068&t=490s How long since you have tried tools? I think there has been some pretty astounding progress during the last couple of months. Until recently i did not use it daily, but now I just cant ignore the efficiency boost it gives me. There are definitely security concerns, and at this point you should not trust code that you do not read/understand, but tbh. i'm starting to believe that AI might (at least in the short term) free up resources to patch stuff and implement security features, that otherwise was not prioritised before due to focus on feature development. What it does to the IT sector in the long run - who knows...

  • source
  • parent
  • hideshow 2 child comments
  • [–] 1 point 1 year ago (1 child)

    That video showed him saying that it's good for autocomplete. But speaking from experience testing it on Rust, Python, JS, HTML and CSS, it performed the worst on Rust. It wrote tests well, but sucked at features or refactoring. Whether the problem is between the chair and the screen, I don't know.

    Whether AI will be able to write secure code, I dunno, I haven't tried. It could be put into the rules to consider security and add tests relating to security or add an adversarial agent that tries to find flaws in the code which can be exploited. That could probably do more than a developer who has no time assigned to care about testing, much less security.

    What it does to the IT sector in the long run - who knows…

    Agreed. Things are moving so quickly, it's impossible to predict. There are lots of people on LinkedIn screaming about obsoletion of humans or other bold claims, but to me they are like drunk fortune tellers: tell enough fortunes and one is bound to be right.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 1 year ago

    My naive hope is that local models or maybe workplace distributed clusters catch up and the cloud based bubble bursts. I am under the impression, that atm. a big difference as to whether a tool works well or not is more related to how well all the software around the actual llm is constructed. E.g. for discovery - being able to quickly ingest an internet url and access a web index is a big force of the cloud based providers atm. And for coding it has a lot to do with quickly searching and finding the relevant parts of the codebase and evaluate whether the llm has all the required information to correctly perform a task.

  • source
  • parent
  • [–] 28 points 1 year ago (4 children)

    Schrödinger's AI: It's so smart it can build perfect security, but it's too dumb to figure out how to break it.

  • source
  • hideshow 4 child comments
  • [–] 3 points 1 year ago (3 children)

    If there are actually no bugs, can't that create a situation where it's impossible to break it? Not to say this is actually a thing AI can achieve, but it doesn't seem like bad logic.

  • source
  • parent
  • hideshow 3 child comments
  • [–] 8 points 1 year ago

    Even if there's such a thing as a program without bugs, you'd still be overlooking one crucial detail - no matter the method, the end point of cybersecurity has to interface with humans. Humans are SO much easier to hack than computers.

    Let's say you get a phone call from your boss - It's their phone number and their voice, but they sound a bit panicked. "Hey, I'm just about to head into a meeting to close a major deal, but my laptop can't access the server. I need you to set up a temporary password in the next two minutes or we risk losing this deal. No, I don't remember my backup - it's written down in my desk but the meeting is at the client's office."

    You'd be surprised how many people would comply, and all of that can be done by AI right now. It's all about managing risk - there's never going to be a foolproof system.

  • source
  • parent
  • [–] 1 point 1 year ago (1 child)

    Rice's Theorem prevents this... mostly.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 2 points 1 year ago

    Another way of working around Rice's theorem is to search for methods which catch many bugs, without being complete.

    I'd guess that hypothetical AI cybersecurity verification of code would be like that, where there are probably no bugs, but it's not a totally sure thing. But even if you can't have mathematical certainty there are no bugs, that doesn't mean every or most programs verified this way are possible to be exploited.

  • source
  • parent
  • [–] 26 points 1 year ago (1 child)

    Ha ha ha ha ha!

    Oh wait, you're serious. Let me laugh even harder.

    HA HA HA HA HA!

  • source
  • hideshow 1 child comment
  • [–] 20 points 1 year ago

    Not with any of the current models, none of them are concerned with security or scaling.

  • source
  • [–] 15 points 1 year ago

    ahahahaha

    Oh, you're serious. Let me laugh even harder.

    AHAHAHAHA

  • source
  • [–] 15 points 1 year ago

    Ah yes, I'm sure AI just patched that software so that other AI could use that patched software and make things so much more secure. What a brilliant idea from an Ex-CISA head.

  • source
  • [–] 15 points 1 year ago (1 child)

    Because then Security would be non-existent.

  • source
  • hideshow 1 child comment
  • [–] 13 points 1 year ago

    One of the most idiotic takes I've read in a long time

  • source
  • [–] 12 points 1 year ago

    Ron Howard narrator: Actually, they would need more.

  • source
  • [–] 12 points 1 year ago

    Clearly she's never seen AI code.

  • source
  • [–] 11 points 1 year ago

    Fix what code? The code it broke or wrote like shit in the first place?

  • source
  • [–] 9 points 1 year ago (1 child)
  • [–] 13 points 1 year ago

    Is that why she's Ex-CISA? 🤣

  • source
  • [–] 6 points 1 year ago (1 child)

    If an AI can be used for automatic scalable defense, it can also be used offensively. It'll just be another digital arms race between blackhats and everyone else.

  • source
  • hideshow 1 child comment
  • [–] 5 points 1 year ago

    Who is paying her?

  • source
  • [–] 5 points 1 year ago

    The look on her face in the thumbnail matches the title perfectly.

  • source
  • [–] 5 points 1 year ago (1 child)

    I just asked an AI what the minimum wage was in 2003 in the UK and it told me that it was £4.50 and that on a 40 hour work week, that came out to 18k a year... But sure, trust it to write and fix code...

  • source
  • hideshow 1 child comment
  • [–] 5 points 1 year ago (1 child)

    AI might pull her head our of her ass... eventually.

  • source
  • hideshow 1 child comment
  • [–] 5 points 1 year ago

    Because its doing so so well now unattended...

  • source
  • [–] 4 points 1 year ago*

    Easterly said that if cybercrime was a country, it would be the third biggest in the world, just behind the US and China.

  • source
  • [–] 4 points 1 year ago

    Based on my understanding of programming I think they're going to need an extra couple people on the security team because of the Ai's "fixes"

  • source
  • [–] 4 points 1 year ago

    People who say these things clearly have no experience. I spent an hour today trying to get one of the better programming models to parse a response. I gave it the inputs and expected outputs and it COULD not derive functional code until I told it what the implementation needed to be. If it isn't cookie-cutter problems then it just can't predict it's way through it.

  • source
  • [–] 8 points 1 year ago (3 children)

    Except that most risks are from bad leadership decisions. Exhibit A: patches exist for so many vulnerabilities that remain unpatched because of bad business decisions.

    I think in a theoretical sense, she is correct. However, in practice things are much different.

  • source
  • hideshow 3 child comments
  • [–] 10 points 1 year ago (1 child)

    My old job had so many unpatched servers, mostly Linux ones. Because of the general idea that "Linux is safe anyway". And because of how Windows updates would often break critical infrastructure, so they were staggered and phased.

    But we've seen plenty of infected Linux packages since, so it's almost a given there's huge open holes in that security somewhere.

  • source
  • parent
  • hideshow 1 child comment
  • [–] 3 points 1 year ago

    This is like a landlord painting everything white. It'll hide / seem to fix some issues, but probably isn't the fix you're looking for.

  • source
  • [–] 3 points 1 year ago

    At first I thought it was the Beaverton. Had to check the URL.

  • source
  • [–] 2 points 1 year ago

    AI can't even write decent unit tests. How the hell is it going to properly red-team this service?

  • source
  • [–] 3 points 1 year ago

    The one thing I will agree with is that If you ignore the AI part and just focus on the idea of having good software that can find code vulnerabilities, that’s a good idea.

  • source
  • 2 comments in c/programmerhumor@lemmy.ml view that post
    [–] 19 points 1 year ago* (1 child)

    I walked into to work this morning to a dev IM needing permissions. Instead of telling me what the error message said, which would contain the allowance needed, he needed he sent me a screenshot of what chatGPT said he needed. GPT effectively said the solution was to let him assume any privileged access policy that exists.

    I think security teams are fine for a while.

  • source
  • hideshow 1 child comment