Perhaps you missed it, but the second affair that was uncovered was that he was fucking the head of Nvidia's health divison. Who is most certainly his subordinate.

But his public persona IS part of his job. A CEO in today's world isn't paid just to run a company. They are also paid to SELL the company stock. He let his personal life become public which hurt the company enough too warrant his removal.