To be fair, he does have a brain tumor. So I can't imagine he's working with a very rational brain. Honestly I think thats probably what caused him leave the FotA. Especially if it formed in the front of the brain.