No

of course it isn't safe… AI has not been proven safe to use for in anything other than trivial info searches

Not only is it obviously not safe, there is zero evidence that it even works.

NO!

Why is the opening story so vague? Middle school counselor gets an automated alert, calls parent, calls police??? No explanation of what caused the alert? Is this supposed to be presented as a success? Then it says "She believes that the interaction built trust between [counselor] and the family."

And other monitoring systems "which would refer the student to in-school suspension " for... reporting "my mom beat me with a pole" or "my uncle touches me"?? I'm so confused by all of this. I don't use AI at all, and never touched AI therapy. I must be so far out-of-the-loop. None of this makes sense.

Ye of course what could ever go wrong bro

No.

a recent national survey found that 20% of high schoolers have used AI romantically or know someone who has, there’s significant interest in keeping students from emotionally connecting with bots. That even includes a proposed federal law that would force AI companies to remind students that chatbots aren’t real people.

Oh god, that generation is fucked

students talk about their life problems with a llama called Kiwi

You just managed to piss off both southern hemispheres at once.

Is that even a question?

What advocates want to prevent is these bots fueling the loss of social skills because they pull people away from relationships with other people, where they have social accountability, Hiner says.

Yeah this is one of my big worries here. And the last section where it's praised compared to other warning systems for not being punitive is horrible though not surprising. It had already become like that when I was a teen. But also, it doesn't have to be like that, we should be treating teenagers like people who fuck up regularly, especially when they're going through shit.

All said though, I do think we need a non Ai way to encourage teens to feel comfortable reaching out for help. I know I didn't trust counselors with my home issues or even consider talking to them about it. Mostly because I knew they'd just make it worse.

Brittani Phillips checked her phone. A middle school counselor in Putnam county, Florida, Phillips receives messages from an artificial intelligence-enabled therapy platform that students use during nonschool hours. It flags when a student may be at risk for harming themself or others based on what the student types into a chat.

Phillips saw that this was a “severe” alert for an eighth-grader.

So, Phillips spent her evening on the phone with the student’s mom, asking questions to figure out what was going on and how vulnerable the student was. Phillips also called the police

Jesus Christ. Was she trying to kill him??