▲ 560 ▼ Based on this graph, and this graph alone, guess at what time I completely blocked OpenAI crawlers (jlai.lu) submitted 8 months ago by hylobates@jlai.lu to c/selfhosted@lemmy.world 65 comments fedilink hide all child comments I really hope they die soon, this is unbearable…
[–] quick_snail@feddit.nl 3 points 8 months ago (6 children) Y'all need to learn to cache things, shiit permalink fedilink source hideshow 6 child comments replies: [–] poVoq@slrpnk.net 6 points 8 months ago (3 children) This is not how things work on the modern web. Did you just wake up from a 20 year coma? permalink fedilink source parent hideshow 3 child comments replies: [+] FreedomAdvocate@lemmy.net.au 1 point 8 months ago (2 children) [deleted] permalink fedilink source parent hideshow 2 child comments replies: [–] poVoq@slrpnk.net 1 point 8 months ago (1 child) For static sites, yes. To actually protect dynamic sites against AI crawlers, Cloudflare has to do much more than just caching. And besides that, Cloudflare is a huge single point of failure and highly privacy invasive. permalink fedilink source parent hideshow 1 child comment load more comments (1 reply) [–] hylobates@jlai.lu [S] 1 point 8 months ago (1 child) There is cache but for some reason this does not affect every page I serve currently permalink fedilink source parent hideshow 1 child comment replies: [–] quick_snail@feddit.nl 3 points 8 months ago Fix the cache. permalink fedilink source parent
[–] poVoq@slrpnk.net 6 points 8 months ago (3 children) This is not how things work on the modern web. Did you just wake up from a 20 year coma? permalink fedilink source parent hideshow 3 child comments replies: [+] FreedomAdvocate@lemmy.net.au 1 point 8 months ago (2 children) [deleted] permalink fedilink source parent hideshow 2 child comments replies: [–] poVoq@slrpnk.net 1 point 8 months ago (1 child) For static sites, yes. To actually protect dynamic sites against AI crawlers, Cloudflare has to do much more than just caching. And besides that, Cloudflare is a huge single point of failure and highly privacy invasive. permalink fedilink source parent hideshow 1 child comment load more comments (1 reply)
[+] FreedomAdvocate@lemmy.net.au 1 point 8 months ago (2 children) [deleted] permalink fedilink source parent hideshow 2 child comments replies: [–] poVoq@slrpnk.net 1 point 8 months ago (1 child) For static sites, yes. To actually protect dynamic sites against AI crawlers, Cloudflare has to do much more than just caching. And besides that, Cloudflare is a huge single point of failure and highly privacy invasive. permalink fedilink source parent hideshow 1 child comment load more comments (1 reply)
[–] poVoq@slrpnk.net 1 point 8 months ago (1 child) For static sites, yes. To actually protect dynamic sites against AI crawlers, Cloudflare has to do much more than just caching. And besides that, Cloudflare is a huge single point of failure and highly privacy invasive. permalink fedilink source parent hideshow 1 child comment load more comments (1 reply)
[–] hylobates@jlai.lu [S] 1 point 8 months ago (1 child) There is cache but for some reason this does not affect every page I serve currently permalink fedilink source parent hideshow 1 child comment replies: [–] quick_snail@feddit.nl 3 points 8 months ago Fix the cache. permalink fedilink source parent