I’ve seen cases where SMART returns clearly impossible data. Not incorrect or oddly formatted, numbers that just can’t be. Also, if a badblocks run returns bad blocks

It's 2017, I'm just an idiot :) Thanks for pointing that out, I have corrected it.

As someone who read most of the site then thank you for doing this.

This is great work, but a quick check: in the body you say the period is 2005-2017, but the title says 2005-2007. Which one is correct?

Good read. The "burned discs die faster" point is the one where a bit of practice goes a long way, because most of it is in your control: buy decent media rather than the cheapest spindle, burn at a moderate speed instead of max, store discs upright in cases away from light and heat, and verify right after burning (most burning tools have a verify option, or just checksum the files on the disc against the originals).

For anything you really care about, keep the files on a drive as well and treat the disc as the copy you can hold, which seems to be the spirit of the article anyway.

Everything they said is on fuckin point.

Yo, thanks 😃

looks like https://github.com/smutscrape/smutscrape

I mean we already do.

Digital images have used compression since forever.

The usual formats are all about compression choices. Jpg, png, tiff, webp, etc

https://shotkit.com/image-file-formats/

Use JPEG XL if you really have to, it's modern and good.

You are probably doing it without noticing. JPEG is a compression standard. you can take a png and turn it into a jpeg only half its size if your lucky. And in case you like simple line art, SVGs have even smaller filesizes.

If you've got a better suggestion, please feel free to post it. This is what I've actually done for scanning books that was tolerable but I definitely do not know everything that's out there.

The other main free OCR software I know of is Tesseract and it needs much better scans and more preparation to get good quality out of. It does not handle italics in the current version and struggled with the tables I needed to work with when I tried it. It is faster when it works, but also frequently makes mistakes that need cleanup like outputting ¥ instead of Y (or vice-versa). I've made heavy use of it in automated pipelines before, but a local LLM + a little scripting is way easier to get decent results out of.

OCR software is out there and usually accessible for free. Like Android, Mac OS, and Windows all have them incorporated somewhere.

I've done this successfully twice with local LLMs. One of the cases was a book of data that I needed to be able to be able to pull examples from quickly, and the other was a long printed prose document that I didn't have a digital copy of but wanted to be able to reference on my kindle. The book of data was much trickier to transcribe overall (multi-column layout + text that was more prone to hallucination/typo output + entries split across pages in a way that was awkward to accumulate), but both had some challenges.

The general workflow was:

• actually scan the documents
• pass each scanned image through the LLM with several prompts, logging the results
• do a merge pass (per page) on the LLM's results including he final formatting I needed
• concatenate/merge the per-page results into a single document
• final processing (e.g. Markdown -> HTML -> ebook with calibre for the prose document)

I did a combination of automated and manual verification + some manual cleanup. In both cases the results likely are not perfect transcriptions, but were good enough for my needs.

I wrote a few scripts to run scanimage on Linux with the appropriate parameters to save into automatically numbered sequential images as I turned pages -- separating evens and odds for one of them since the book did not fit nicely on my scanner. That simplified the process of scanning a lot, but it was still quite tedious. I automatically displayed the image after each scan and had a flag to rescan the last page -- which I needed to do occasionally.

It took some experimentation to get the prompts right and I don't have a complete, easy-to-use process to share; what I've got right now needs tweaking per book/document.

Some of the prompts were things like this though:

Transcribe the text on page {{page_number}}. Do not include headers (e.g. author, book title, page number, etc).

and

Which words (if any) are italicized on page {{page_number}}? Output the word/snippet -- plus context. This may include individual words, phrases, or entire sentences. Look carefully.

Having it convert to Markdown was helpful to get styling although it wasn't always consistent about it. I had better results using simple prompts and multiple passes than trying to get it to do things all at once in a single detailed prompt.

I set a thinking budget (1000~2000 tokens depending on task) and used a low temperature. I got about ~100 pages an hour on the simpler prose document (per pass) with Qwen 3.6 at Q6_K quant on an R9700 GPU. I suspect you could probably do similar with Gemma4 on a lower VRAM GPU if needed.

No link?

How long until these are all banned or blocked by isps ? And illegal to use a VPN . I think the corpos are catching on that we dont give a fuck about new games and they will come after Roms.

It should work. I haven't done that but I've been using Blu-ray drives with Linux for like a decade now and haven't had any trouble reading and writing discs

Biggest public rom archive.

You're giving me mixed signals, Jason

Biggest rom archive on the web.

Is that a sexual advance or a threat of violence? I don't know what myrient is and I'm not sure if it's good or bad to be put onto the Internet Archive.

https://fmhy.net/gaming#rom-sites

Various websites hosting individual romsets, a good chunk of it is on archive.org, there are projects like mariocube, basically a lot of different places depending on what you're after. And there are some private invite-only data troves the scale of myrient as well.

Of note, the preservation scene uses dat files to distribute metadata about the data in question, allowing managers like ROMVault to take a pile of files and rename/move them into a file structure designating what they are and confirming that they're undamaged. If you're going to start trying to collect, look into doing it this way, rather than trying to just mirror someone else's work.

What is the less convenient way to access it?

Just a reminder: this was never the sole source of any data. It does not need to be backed up. If you're genuinely interested in preservation, try not to get swept up in the hype around this site going down, because there's a mess of confusion being spread by people who mean well but do not understand the situation.

This does suck, it was a good easily-accessible public source of this data, but only convenience is lost.