Yeah, there's a reason only a handful of companies on the planet even try it. On top of the time and technology requirements, there's also a massive patent minefield to navigate. Qualcomm is something like 20 times bigger than Valve with chips as their entire business model, and even they're still not big enough. Qualcomm contracts TMSC and Samsung to use their fabs.

Edit: added a bit...