The very nature of how it functions is unreliable. It's a statistical probabilistic model. It's great for what it was designed to do but imagining that it has any way of rationalising data is purely that, just imagination. Even if let's say we accept that it makes an error rate at the same rate as humans do (if it can even identify an error reliably), there's no accountability in place that ensures that it would check the correctness like a human would.

You could say that. But you could also say that none of these other technological advances got pushed through this badly while being obviously not ready for widespread use.

And also, can you really say that though? Most other technological advances had a pretty clear distinction from the older way of doing things.