Someone brought the HTtrack app to my attention in this thread. Superficially it’s a great concept. But it did not work for me.

The author of HTtrack says they respect the robots.txt files. I’m not sure if this is my problem. But what an absurd line to draw. Why should a realtime GUI user have more privilege to view a website than a CLI user?

If anything, it should be the other way around.

• People who lack the priviledge of having Internet at home face this nasty discrimination of being treated like a bot after they make the extra effort of commuting to a public library just to fetch a website for offline viewing later. This 2nd-classing of a demographic who is already marginalized is quite despicable.
• People with Internet at home can schedule HTtrack to run at an off-peak time and on a narrow bandwidth put less burden on the server than the realtime GUI users who obviously hit the site mostly during peak times.
• (update) Some people are on measured rate Internet connections that give tiny daytime quotas and generous late night quotas. Tools like HTtrack are needed to manage this. Which ultimately benefits the more privileged Internet users who have no constraints.

Update

There is a poorly worded -s0 option to ignore robots.txt. Fooled some people into thinking the tool uses robots.txt to direct the fetches.

Shopping for airfare is clearly a game full of shenanigans. You find a cheap ticket, get a (likely fake) indicator warning how few seats are available with a countdown timer, rush through a lengthy process of being forced to make a shit-ton of decisions like whether you want to buy an neck wrap, selfie stick, a bad travel insurance deal, .. lots of shit to get through to slow you down. You finally get to the last screen and it says “price has increased since we first quoted you”. Motherfuckers.

Considering aircraft are quite shitty for climate, why not make the airlines shenanigans backfire against them? We have bots arbitrarily hit the air travel sites, enter bogus orders but never submit the last page. Just give the airlines a false indicator of demand. This games their dynamic pricing to quote prices higher than optimum. Any deviation from optimum prices translates into lower profit, likely a consequence of lower sales.

Perhaps an org like Greenpeace would be interested in this tactic.

cross-posted from: https://lemmy.sdf.org/post/48532948

Banks are on an unstoppable uncontrolled trajectory in pursuit of KYC over-achievement. That is, they over-collect far more data on people than legally required (before it gets leaked to criminals in data breaches). Banks’ privacy policies are rife with anti-consumer weasel words.

It’s such a shit-show that privacy proponents have no real choice other than to quit banks and operate entirely with cash. Not many people have that level of discipline.

Software can turn this situation around. For example, there are ~6000 privacy-abusing banks and credit unions in the US. If a robot harvests all the privacy policies, fetches AOS apps to check permission reqs, records those with websites MitMd by Cloudflare, and uses all that info to find the lesser of evils, consumers can participate in creating a competition for privacy (as opposed to a competition of meaningless soul-selling fractions of a percent of interest earnings). The heart of the problem is banks are only getting pressure from the side of oppressors and tyranny and no pressure from the side of the people they purport to serve. Software and data can remedy this.

Worth noting that long before the AI bubble started, a university in the US studied bank privacy policies in bulk using a scraper bot that just looked at the standardised privacy disclosure forms for which all banks must conform to a standard layout. The data has rotted by now so their research is not of much use.