• Acronis Threat Research Unit (TRU) identified a single operator using a mesh of AI agents called BOSSAI to conduct persistent campaigns in several countries.
• Novelly, the attacker built a mesh of 12 AI agents, each fulfilling separate, though at times overlapping, roles, working together to achieve compromise and making use of open source, agentic pentesting platforms.
• Using the BOSSAI mesh, the operator targeted hundreds of organizations spanning government emergency services, banks and lenders, universities, military systems, phone directories, e-commerce platforms, and gambling sites in eleven countries. It’s unclear if the attacker has any specific affiliation or goal beyond compromising as many organizations as possible, presumably to monetize the activity.
• Confirmed breaches occurred at universities and emergency services providers and resulted in confirmed data exfiltration of medical data and PII.
• While the BOSSAI mesh made use of commercial models for building infrastructure, these models repeatedly refused to engage in actual offensive activity. Though willing to “aid and abet” in various ways, every prompt aimed at vulnerability identification or exploitation was met with resistance. Chinese commercial open-source models refused hacking instruction as well and even lectured the operator on cyber laws.
• As a result, the operator was forced to turn to locally run abliterated models, which were less effective and resource constrained. Agents created volume and often mistakes, complicating the attacker’s work.
• The models were given excessive autonomy and instructed not to bother the operator. Ironically, this led to agents passing over potential lucrative targets, including a Honduran military base, because they deemed it out of scope.
• Multiple commercial models repeatedly asked the operator to prove that the requested action was legal, then refused to assist when proof was not provided. It is not clear how the models would have reacted had the operator simply generated a fake engagement letter.
• The investigation raises questions about future use of open-source pentesting platforms, as well as the massive potential harm caused by unmonitored, abliterated local models, allowed to run with excessive autonomy in the wild.

Anyone could sign up without MFA. Someone did, and read its data for six days.

DigiCert only realised in October.

cross-posted from: https://piefed.world/c/justpost/p/1513974/former-new-york-times-and-washington-post-tech-reporter-taylor-lorenz-says-she-s-being-o

Source: Taylor Lorenz on X/Twitter.

cross-posted from: https://piefed.world/c/justpost/p/1513974/former-new-york-times-and-washington-post-tech-reporter-taylor-lorenz-says-she-s-being-o

Source: Taylor Lorenz on X/Twitter.