The Department of Homeland Security, which oversees FEMA, confirmed Thursday that Phillips is leaving the agency, saying he is taking leave for personal reasons. But sources tell CNN the departure was not voluntary: New DHS leadership had grown weary of the embarrassment surrounding Phillips’ public image and of his periodic clashes with the department’s other leaders.

Even so, Phillips had won over some career FEMA officials. When he arrived, then-Homeland Security Secretary Kristi Noem and her de facto chief of staff Corey Lewandowski were taking aggressive steps to rein in FEMA, shrink the agency and bottleneck funding.

FEMA officials said Phillips stood out because he was willing to resist moves that would potentially hobble the agency’s work.