that someone who owns the AI/Drones will use it to kill us.

Or that they'll just be lazy and sloppy.

In movies you used to sometimes see someone put a brick on a car's gas pedal so they could send it through a wall, or into a lake, or whatever. In a comedy, you sometimes used to see someone rig up something so that their car could drive while they slept.

Running AI "agents" is basically like opening up some application that has a lot of keyboard shortcuts to run various scripts and then putting a brick on the keyboard.

Unfortunately, we're now at a stage where some people think that's a good idea. The machines aren't capable of thought, they have no desires or goals. But, if some of those scripts instruct a drone to go out and kill someone, the brick on the keyboard can get someone killed.