699
School shooting survivor sues AI gun detection firm after system failed to spot weapon (arstechnica.com)
submitted 5 months ago by to c/technology@lemmy.world
89 comments fedilink hide all child comments
 

The injured teenage survivor of a January 2025 shooting at a Nashville, Tennessee high school recently sued the manufacturer of an “AI gun detection” system that failed to detect the handgun that left two dead, including the shooter.

According to the lawsuit, which was filed in Davidson County court last month, the security company Omnilert either knew or should have known that there were “significant operational limitations in its gun detection system that could result in detection failures during actual emergencies, including limitations based on camera placement, proximity of the weapon to camera sensors, camera angle, lighting, and weapon visibility.”

Omnilert cofounder Ara Bagdasarian declined Ars’ invitation to answer questions about the lawsuit. System Integrations, the other defendant in the case, which resold the Omnilert system, also did not respond to Ars’ request for comment.

you are viewing a single comment's thread
view the rest of the comments
[–] 2 points 5 months ago (3 children)

The cheap system I have with a Google Coral and FOSS software

I'm guessing you're using Frigate?

Having such systems as a later if defense is good. As the only defense, not so much.

Agreed. The system I had developed was built explicitly as a human-in-the-loop system. It never made any decisions on its own. It was just a tool to enable the existing security staff to have better visibility. That's it.

You can make whatever argument you want about viability and efficacy. The only point I'm making is that our system was just an additional tool for security to use; not the only one.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 3 child comments
    •
  • [–] 2 points 5 months ago (2 children)

    Yup. I wasn't disagreeing so much as pointing out a common failing with AI adoption in general: the number of cases where it's being implement as a replacement for existing functional systems or humans rather than an augment to them. The "additional tool" aspect is 100% my preferred use case (where it functionally makes sense), but many are seeing it as a human-replacement that feeds into their desire for control/subservience, cost-reduction, or rent-seeking.

    Local-AI actually has a lot of useful cases, and AI vision is something that in many forms has been around for awhile and is generally fairly effective. It's great as a tool for indexing or enriching visual data that would otherwise at best a slog and at worst improbable for a human. Surveillance video is especially good with this. I see it as an addition to motion detection. Instead of needing to go through hours of footage of video, you only need to go through stuff where movement was detected, and from that you could further search for "frames tagged as having a person/bear/vehicle/whatever". The thing is, I'm not depending on it to protect me from running into a bear when I go out out my trash (although Frigate can do alerting), but I could use it to follow how the bear moved between cameras/zones through my property and possibly use that to better bear-proof the premesi

    In this particular case, it might be somewhat useful as an early warning if it can actively detect human+firearm, but there were obviously going to be either a number of false positives or negatives that make it less valuable as such. It could still be useful in reconstruction or actively following some incidents, but we need to keep humans in the loop too.

    AI isn't bad/evil or good when used correctly and with knowledge of the limitations in a given use-case. Trusting AI to replace humans/judgement is often very bad, and massive datacenters are obviously a major source of concern, but the issue is again in the use not the overall technology. Just like nuclear fission can produce huge amounts of power to either keep the lights on or blow stuff up, AI can help sort through your album if 5000 photos to find that one shot in a jiffy... or it can be used identify, track and you for a surveillance-state, and false positives in the latter case can make for a very bad day.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 2 child comments
    •
  • [–] 1 point 5 months ago (1 child)

    but there were obviously going to be either a number of false positives or negatives that make it less valuable as such.

    It really comes down to the specifics of the technology and how it's implemented. The system I worked on was astonishingly good. Even the local police of a particular installment wanted to do a test of the system (basically walk around with various guns of all different sizes) and they were stunned at how well it worked.

    The funny thing was that we built the system to work 100% locally, and we even insisted on air-gapped networks (but wasn't a requirement). The amount of people and companies who asked if we could connect the system to the internet for easier access was worrying. Whenever we tried to explain the basics of data security and the potential issues we were just looked at like we were nuts.

  • permalink
  • fedilink
  • source
  • parent
  • hideshow 1 child comment
    •
  • [–] 1 point 5 months ago

    Yeah, cameras should almost always be on a segregated network, preferably with port-security. The NVR may need to connect to the Internet for updates - assuming one can't just upload that via the UI - but that's about it.

  • permalink
  • fedilink
  • source
  • parent