If they don't crawl the entire network then most of the network cannot contact them at all? Which makes it ... not really a network. That's where federation would come in.