Power corrupts
Magical thinking helps no one. Power is not a magic force that corrupts all who touch it, this isn't Lord of the Rings. It is a social relation that allows people to more effectively work in their interests. The issue is that society in much of the world thus far has been structured such that a minority class, be they the landlords and aristocracy of feudalism or the bourgeoisie of capitalism, hold an effective monopoly. They use this monopoly to extract surplus and protect their property. The solution is not to wish away power, but to seize it and democratize it, transforming it from a tool of class exploitation into an instrument of social administration.
subjecting a healthy adult to anyone other than themselves is always harm.
By your logic, nuclear power station workers being subject to the safety team is truly harmful. An operating room being subject to a chief surgeon is another great harm.
Or is it possible that hierarchy is a useful social relation that has been reproduced across geography, centuries, and even species for a reason?
In nature, coordinated structures emerge because they solve material problems of survival and resource management. In human society, as the division of labor grows more complex, large scale production physically demands coordination, discipline, and unified direction. A factory, a railway network, or a hospital cannot run on pure consensus without collapsing.
The goal should not be to abolish all authority, which would mean abolishing industry and modern society itself. The goal should be to abolish class authority. End the subjection of people to people as exploiters, and replace it with the rational, democratic administration of things and the technical direction of production.