▲ 461 ▼ This website is for humans (localghost.dev) submitted 1 year ago by MarcellusDrum@lemmy.ml to c/technology@lemmy.world 60 comments fedilink hide all child comments
[–] aarmea@lemmy.world 8 points 1 year ago (2 children) https://localghost.dev/robots.txt User-Agent: * Allow: / permalink fedilink source hideshow 2 child comments replies: [–] trk@aussie.zone 6 points 1 year ago (1 child) Is there a more ignored file on the internet than robots.txt? permalink fedilink source parent hideshow 1 child comment replies: [–] ohshit604@sh.itjust.works 5 points 1 year ago Do dnt headers count? permalink fedilink source parent
[–] trk@aussie.zone 6 points 1 year ago (1 child) Is there a more ignored file on the internet than robots.txt? permalink fedilink source parent hideshow 1 child comment replies: [–] ohshit604@sh.itjust.works 5 points 1 year ago Do dnt headers count? permalink fedilink source parent
[–] ohshit604@sh.itjust.works 5 points 1 year ago Do dnt headers count? permalink fedilink source parent