Why should OpenAI be held responsible for this particular incident? Because their LLM recommended the user to contact a suicide hotline when he expressed his suicidal thoughts? Because it correctly answered his questions and confirmed that the safety of his shotgun was off? Don't get me wrong, I like having companies held accountable as much as the next guy, but why don't you ask the gun production companies to be held responsible as well? Hell, we might as well be asking for his parents to be held accountable for their offspring's actions.

Yes, LLM's can, with or without external manipulation, provide information about sensitive subjects, which can then be consumed by vulnerable individuals who may act in a harmful way. That individual would have acted and reached the same outcome not matter if the LLM would have existed or not, because there are multiple options to using an LMM to gather information.

What are the realistic expectations of OpenAI to implement to prevent these things, that doesn't automatically mean stricter restrictions to the user data privacy and further strengthen the control of citizens (thanks to the flourishing relationship between OpenAI and the American government)?