Per the report, the messages the woman sent were apparently flagged by Anthropic’s automated safety monitoring system, which pings human reviewers any time a message meets a certain risk threshold. Humans reportedly took a look at these specific messages and decided they were serious enough that they should alert law enforcement, and they did.

Imagine hearing about search engines reviewing your queries and forwarding them to law enforcement. I feel like we'd see a much different response.