Agreed. It's a bit of both. There's been people talking about the "alignment problem" in the right fora for like 20 years. And, if we ever have consciousness / sentience that is created by us, it will be something that it would be nice to have an answer to.

But, the most relevant AI issues today are not that. It's about liability-shifting and other externalization by the powerful, misunderstanding and misuse by the well-meaning, and various cognitive hazards. These are not marketing, these are what the companies don't want you to think about. There also lock-in and labor issues that the model providers don't want to talk about, though these do have somewhat-known mitigations. You get plenty of people that are primarily concerned with these issues being dismissed a "AI doomers" which in, I think, wrong.

Superhuman Intelligence might be a threat if any of the companies developing AI right now had any way that their word guessing programs could possibly simulate consciousness. The best argument I've heard so far that they could is "no one understands how LLMs work, and no one understands how the human brain works, so you can't prove they're not the same."

Talking about superhuman intelligence now, and fretting and worrying about what we're going to do in the future where AI is smart enough to be dangerous, ignores all of the ways AI is dangerous now, like giving wronf answers, making it almost impossible to find useful information on the internet because sites that have it are buried under AI-generated SEO slop that's wrong half the time, consuming way more electricity and water than municipalities can afford to supply them with, the datacenter heat island effect, carbon emissions and other forms of deadly pollution from brand new power plants for AI datacenters, to say nothing of all the jobs that are being destroyed by bosses that think their customers won't notice the change in product quality.