What always feels weird to me, is that we don't need that inflection point of AI turning superintelligent or whatever. Already today, we're producing drones that do autonomous killing, where we just need to scale up production.

I guess, on the plus(?) side, the same is true for bombs and we have not wiped everyone out yet with those either.

That is, I guess, where the inflection point comes in. Killing humans is ultimately quite pointless from a human perspective. But an AI with no understanding of this pointlessness might coordinate the killing with no restraint.
Or well, as the blog post mentions, it might see wiping out humanity as the solution to climate change, or as a way to end violence between humans etc..

Current AI iterations certainly prove to be much better at doing things than deciding whether these things make sense.

because if you really believed this you’d be bombing datacenters instead of posting on Twitter.

No. That wouldn't stop AGI - only delay it at best. That's like pouring water on every person rubbing two stick together thinking it's going to stop the invention of fire.

This is why AI researchers care so much about “alignment”: building AIs that share genuinely human beliefs and values. If we build a “misaligned” superpowerful AI — an AI with goals that are alien to us — it might sweep humanity away. It could kill everyone deliberately, e.g. to stop us getting in the way of some goal. It could kill everyone in passing, e.g. like we might pave over an anthill to build a road. Either way, everyone dies.

Ah so they think AI will kill us because capitalism fundamentally does not value human life and thus AI cannot currently be aligned with human beliefs and values because it is funded by shareholder greed.

That makes more sense now. Too bad no one is gonna fix the larger problem to prevent this. Might as well summon the horsemen now