this post was submitted on 27 Oct 2026

138 points (99.3% liked)

Technology

technology@lemmy.world

89,543 readers

13,910 active

2,480 users here now

This is a most excellent place for technology news and articles.

Our Rules

Follow the lemmy.world rules.
Only tech related news or articles.
Be excellent to each other!
Mod approved content bots can post up to 10 articles per day.
Threads asking for personal tech support may be deleted.
Politics threads may be removed.
No memes allowed as posts, OK to post as comments.
Only approved bots from the list below, this includes using AI responses and summaries. To ask if your bot can be added please contact a mod.
Check for duplicates before posting, duplicates may be removed
Accounts 7 days and younger will have their posts automatically removed.

Approved Bots

founded 3 years ago

moderators

L3s@lemmy.world

enu@lemmy.world

technopagan@lemmy.world

L4s@lemmy.world

L3s@hackingne.ws

Discord will soon let anyone find public server messages through Google (discord.com)

submitted 1 week ago by beep@piefed.world to c/technology@lemmy.world

39 comments fedilink hide all child comments

you are viewing a single comment's thread
view the rest of the comments

[–] AppleMango@lemmy.world 12 points 1 week ago* (8 children)

Now lemmy servers need to do the same.

Maybe some do, but they never show up in search queries whenever I tried. I even filtered by sites.

hideshow 8 child comments

[–] QuadratureSurfer@piefed.social 7 points 1 week ago (2 children)

I see Lemmy/Piefed results through DuckDuckGo.

parent

hideshow 2 child comments

[–] AppleMango@lemmy.world 3 points 1 week ago (1 child)

I'm using DuckDuckGo too, which server are you seeing on the results? I'll try to filter specifically for that in the future.

parent

hideshow 1 child comment

[–] QuadratureSurfer@piefed.social 5 points 1 week ago

PieFed.social, but I've occasionally seen some lemmy.world ones as well.

For example, just try searching for: "starcitizen steamdeck guide"

And third result currently goes to PieFed (a PieFed.social account posting to a Lemmy.world community).

parent

[–] miadidi@lemmy.world 6 points 1 week ago (4 children)

A lot of lemmy servers have Human Verification in front of them maybe that’s why?

parent

hideshow 4 child comments

[–] AppleMango@lemmy.world 3 points 1 week ago (3 children)

Could whitelist specific search engine crawlers

parent

hideshow 3 child comments

[–] diaphragmwp@discuss.tchncs.de 3 points 1 week ago (2 children)

The bots will pretend to be search engine crawlers. And not all have their own IP ranges.

parent

hideshow 2 child comments

[–] Sharks@pawb.social 3 points 1 week ago* (last edited 1 week ago)

You can't impersonate search engine crawlers, because the server can verify that it's really a search engine crawler

See this from google: https://developers.google.com/crawling/docs/crawlers-fetchers/verify-google-requests

Here, duckduckgo: https://duckduckgo.com/duckduckgo-help-pages/results/duckduckbot

parent

[–] AppleMango@lemmy.world 2 points 1 week ago

A lot of article websites whitelist Google's crawler, right? Can't that be done on a domain-basis? Or maybe have some kind of authentication-based system 🤔, I guess that would require the crawlers to be refactored though

parent