I'm sure people have been making crawlers for 30 years... But I haven't been, so I've been able to access "knowledge" and shape it for my use case. I think equating ~5 queries to torching an acre of forest is a bit hyperbolic which doesn't help anyone.