gravitywell comments on AI crawlers user-agent strings?

submitted 1 year ago by m33@lemmy.zip to c/ai_infosec@infosec.pub

3 comments fedilink hide all child comments

Any recommandations for an actively maintained AI user-agent list please ?

This is not perfect but I use that trick to block or slow down some of them... If you have a better way I'm all ears.

you are viewing a single comment's thread
view the rest of the comments

[–] gravitywell@sh.itjust.works 3 points 1 year ago

I just came up with a list on my own from looking at nginx logs

Amazonbot
Amazon-Bot
Claudebot
SemrushBot
AhrefsBot
MJ12bot
DotBot
BLEXBot
bingbot
GPTBot
meta-externalagent