How does it differentiate an "AI crawler", from any other crawler? Search engine crawler? Someone monitoring data to offer statistics? Archiving?

This is not good. They are most likely doing the crawling themselves and them selling the data to the best bidder. That bidder could obviously be openAI for all we know.

They just know that introducing the sentence "this is anti AI" a lot of people is not going to question anything.