I feel like GPU is massively parallel but still somewhat sync, but I'm not sure.

But what I think about the article is that it mixes a lot of wishful thinking into reality. There were other CPU architectures, including ones without synchronisation, and most of them never left research. In part because current architecture is abundant, but I think that also because making something complicated work async is quite hard.