[–] 2 points 4 days ago

I noticed you started another rewrite of Iocaine for version 4.x. May I ask about plans for version four compared to version three?

I wrote a bit about the plans over on my blog. The major changes in a nutshell are:

  • Nam-Shub of Enki elevated to "default script" status
    • It also becomes a module library, and will be available in both Roto and Lua
  • Roto upgraded from 0.9 to 0.13 (bringing a whole lot of nice language features, more performance, and less code I need to write)
  • Half the dependencies axed.
  • Plugins for Caddy, nginx, and Apache, embedding iocaine.
  • Much, much, much better error reporting (see an example of an error here)
  • Throttling & rate limiting

How well does Iocaine integrate with OpenRC? I don’t use SystemD much, as I tend to run Artix.

Pretty well, I think? There's an OpenRC init script in the repo. I haven't tested it myself, I currently use systemd on Linux, and NetBSD's init system or dinit on NetBSD. The only systemd-specific thing in iocaine is that it notifies systemd when it finished starting up - and that's going to go away in iocaine 4, because I'm cutting dependencies, and this tiny thing isn't worth a dependency.

  • source
  • parent
  • context
  • [–] 8 points 1 week ago (2 children)

    FWIW, while iocaine does generate copious amounts of garbage, it is never going to poison LLM training data: it's trivial to filter iocaine-generated garbage out pre-training. The garbage is there for two reasons:

    1. To serve the crawlers something.
    2. To poison the crawling queue.

    Not the models, the crawling queue. Place an identifiable substring into every generated URL, and you can catch any crawler that works off of a queue collected by cheaper crawlers.

    It's not really a poisoning tool - it originally started with that goal, but that goal has changed a loooong time ago -, it's more bot detection & mitigation nowadays, the garbage is just one of the tools it achieves this goal with.

    But while I'm here, I'd like to mention Alun Jones' fakejpeg (or my rust port): a library that can help you generate structurally legit-ish JPEGs, files that most decoders will happily display, but with entirely random content. Generating them is very cheap. How effective it is in poisoning? I don't know. But I find great delight in looking at the random noise.

    Hello Lemmy

  • source
  • [–] 3 points 3 weeks ago

    Most crawlers don't care - they're purely downloading. Most garbage will also be filtered out before training, and generating plausible garbage is expensive - not really worth the trouble. What you can do with the garbage, is that you can place unique identifiers in the links you serve, so when a crawler that you can't recognize otherwise comes visiting a poisoned URL, you can firewall it off. This works because the crawlers use a queue: they get a list of URLs to visit, they download them, and later on, the URLs from those get extracted, and crawlers will visit those URLs.

    So if you manage to trap a bot, and overwhelm its queue with poisoned URLs, you can also catch bots that you wouldn't be able to easily identify otherwise. And that is why serving an endless maze of garbage is so very useful. Because it lets you catch stuff like headless chromes operating through residential proxies: because those too, follow a queue.

  • source
  • parent
  • context
  • [–] 2 points 2 months ago

    Way back when, my Dad always told me to have at least three copies of everything, because one backup is no backup. So I have a 1TB Storage Box at Hetzner (€4/month, in a different datacenter than my VPSes) I push backups to with Restic. I also backup to a local machine (also with restic), and to my Dad's NAS (you guessed, with restic), and to my old Debian server until I decommission it (also with restic).

    All legs have different retention settings, based on how much space I have at a particular backup site.

  • source
  • [–] 6 points 4 months ago

    It was the night of December 24th, 1996. I turned on the family PC, then running Win95, and found my D:\ drive corrupted. Windows had no tools nor docs how to resurrect a corrupted filesystem. I cried, and two days later installed SuSE on a spare disk.

    Some 20 years later, I restored about half of the disk lost in 1996, because Linux had the tools, and the docs, and encouraged me to learn.

  • source
  • [–] 5 points 4 months ago (1 child)

    I'm running Tang on a VPS, outside of my homelab. Servers in my homelab set up networking and a dedicated WireGuard tunnel to the VPS from initrd, to be able to talk to Tang, to help unlock the filesystem. The WireGuard tunnel is only allowed from my home ISP's ASN. So if anyone picks up all my equipment from my homelab and walks away with them, they will not be able to boot them up, unless they connect from my ISP's ASN (good luck), or know the passphrase.

    Additionally, some of my homelab computers that support TPM also have a TPM pin, so walking away with the disk only, and connecting from my ISP's ASN would still not be enough. This is rather pointless, anyone who walks away with the disk only will likely take the entire computer instead. But it was fun setting it up.

    In the not so distant future, I'll update this setup to use Shamir Secret Sharing more, where I'll have three pins: my VPS (via Wireguard), a small computer somewhere else in my apartment, and a third at a neighbour (+ TPM on supporting computers).

  • source
  • view more: next ›