Someone should write an update to the Simple Sabotage Field Manual.

I'm not having trouble with it as such, it's just a slow and painstaking process. The source is crappy enough that an enormous number of characters need to be checked manually, and it's ridiculously time-consuming.

I wonder if they gave considered crowdsourcing this, having many people type in small chunks of the data by hand, doing their own character recognition? Get enough people in and enough overlap and the process would have some built-in error correction.

OCR is mostly good enough. Problem here is we have 76 pages that we need to be read perfectly, with a low fidelity input

We also have very little in the way of error correction, since it's mostly not human readable

Yes, it's base64. And what's behind it could be anything that can be attached to an email.

In this case, it's a PDF. If the base64 text can be extracted accurately, then the PDF that was attached to the email can be recreated.

The challenge is basically twofold:

1. There's a lot of text, and it needs to be extracted perfectly. Even one character being wrong corrupts it and makes it impossible to decode.
2. As the article points out, there are lots of visual problems with the encoded text, including the shitty font it's displayed with, which makes automating the extraction damn near impossible. OCR is very good these days, but this is kind of a perfect example of text that it has trouble with.

As for my approach, I'm basically just slowly and painstakingly running several OCR tools on small bits at a time, merging the resulting outputs, and doing my best to correct mistakes manually.

Sound like your first puzzle is to list out all the meanings of colon and figure out which one of them looks like wheels when turned sideways.

Oh I thought you meant Epstein uses fish.

2^78 is large but computers can do an awful lot per second, so if only about some the pages contain attachments 2^40-55 is something you could bruteforce in weeks if you can do millions of attempts a second

I guess the same way email can have html as an attachment for the same thing a plaintext does, evidently some of these mails suggested a mailer actually pdf encoded the email and attached, as well as the plain text.

So when someone replied with plaintext the base64 encoded PDF that they were replying to got 'quoted', meaning the unredacted email they were replying to is in there, just messy due to font confusion in the provided format.