Brady wasn't everything for the patriots. He was a lot but you can look at seasons like 2008 when Brady tore his ACL and only played part of the first game. Yet the Patriots still went 11-5.
If it had been Brady propping up the entire team I would have expected sub 8-8 record. That said, I couldn't care what happens to them