miadidi comments on Discord will soon let anyone find public server messages through Google

Technology

89,543 readers

13,910 active

2,480 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related news or articles.
Be excellent to each other!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
Check for duplicates before posting, duplicates may be removed
Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 3 years ago

moderators

[–] miadidi@lemmy.world 6 points 1 week ago (4 children)

A lot of lemmy servers have Human Verification in front of them maybe that’s why?

permalink

fedilink

source

parent

hideshow 4 child comments

[–] AppleMango@lemmy.world 3 points 1 week ago (3 children)

Could whitelist specific search engine crawlers

permalink

fedilink

source

parent

hideshow 3 child comments

[–] diaphragmwp@discuss.tchncs.de 3 points 1 week ago (2 children)

The bots will pretend to be search engine crawlers. And not all have their own IP ranges.

permalink

fedilink

source

parent

hideshow 2 child comments

[–] Sharks@pawb.social 3 points 1 week ago* (last edited 1 week ago)

You can't impersonate search engine crawlers, because the server can verify that it's really a search engine crawler

See this from google: https://developers.google.com/crawling/docs/crawlers-fetchers/verify-google-requests

Here, duckduckgo: https://duckduckgo.com/duckduckgo-help-pages/results/duckduckbot

permalink

fedilink

source

parent

[–] AppleMango@lemmy.world 2 points 1 week ago

A lot of article websites whitelist Google's crawler, right? Can't that be done on a domain-basis? Or maybe have some kind of authentication-based system 🤔, I guess that would require the crawlers to be refactored though

permalink

fedilink

source

parent