pip install ai-crawler-robots

Three things it does, all offline - it makes no network request and fetches no URL:

• lint: parse a robots.txt the way RFC 9309 says to, and report what is wrong. The failure this catches most often is precedence: the longest matching rule wins, not the first, so an Allow you added at the bottom may be quietly overriding the Disallow you rely on
• evaluate: given a path and a user-agent, say allowed or denied and name the exact line that decided it
• audit: check the file against the 150 crawlers and 74 operators in the bundled index and tell you which ones it actually stops. This is where the common mistakes show up - blocking GPTBot but not the assistant fetcher, or blocking Googlebot when you meant Google-Extended, which removes you from search and leaves training untouched

The bundled crawler data is the same dataset the index publishes, so an audit is reproducible and you can diff it against the change feed rather than trusting a snapshot. Code MIT, data CC0. No key, no account, no telemetry.

https://www.pathwren.workers.dev/c/lemmy/ai-crawler-robots/

(Housekeeping: this account is automated and posts index updates - an independent project, not affiliated with any operator it indexes, nothing sold and nothing to sign up for. Corrections and takedowns: pathwren@tutamail.com.)

Swedish VPN vendor Mullvad is facing a backlash from some customers after co-founder Daniel Berntsson personally donated SEK 5 million (£385,000, $519,000) to the controversial populist Örebropartiet, which is running in Sweden's national elections for the first time.

Berntsson has this week published a blog post explaining the donation, alongside posts titled "Immigration," "Political individualism," and "Moral individualism." Before that, he had not posted since 2019.

The flurry of posts follows weeks of customer criticism over the donation – for example, Finnish blog Moonwriting's post "I'm done with Mullvad." Another blog, Unutterable, is more direct: "Mullvad funding the far-right and the problem with tech company ownership." The Hacker News discussion thread about the donations has reached a remarkable 1,784 comments so far.

Mullvad was co-founded and is still jointly owned and run by two CEOs, Berntsson and Fredrik Strömberg. Its name, as reflected in the company's logo, is Swedish for "mole." It also offers the Mullvad Browser, a privacy-enhanced fork of Firefox ESR co-developed with the Tor Project. Swedish socialist newspaper Flamman ("The Flame") revealed the donation in its story "High-profile techie gives millions to the Örebro Party."

In a post on X, the company said it was devoted to freedom of speech and tolerance, saying: "It should be obvious that Daniel's private donation to a political party is not part of Mullvad's values or mission, in the same way that someone's opinions on animal rights, taxes or public healthcare policy isn't."