I mean its not an LLM, but its a form of transformer model that is doing the OCR (if you look at the code I put out). Just the OCR component has taken almost 10 days. The model I'm using is incredibly small compared with a full LLM, and it takes almost ~30s a document (see the DB for the actual timings).

Before I kicked this off I did the math on using Claude and open AI to do this. The price would have been between $1.5-2.5k in tokens to do so. So the process I'm using is a lot slower (I could have done this in hours leveraging Anthropics or OpenAI's GPU's instead of rolling my own), but its also OCR specific and frankly, other than the failures I've mentioned previously (very small text), I'm yet to find any substantial mistakes or hallucinations. Its at least 99% accurate.

But beyond that, I'm absolutely planning on using an LLM to go further than I've done already.

Specifically, I'm going to run this prompt:

prompt = Extract ALL contact information from this document. ONLY extract what exists EXACTLY as written - DO NOT make anything up. CRITICAL: Extract EVERY person mentioned (full names, nicknames, first names only, Mr/Ms + lastname). Look in headers AND body text.

Document: {content}

Return ONLY JSON: {{"emails": [{{"email": "exact", "context": "where", "associated_name": null}}], "names": [{{"name": "exact", "associated_email": null, "associated_phone": null, "context": "where"}}], "phone_numbers": [{{"phone": "exact", "associated_name": null, "context": "where"}}], "organizations": [{{"name": "exact", "context": "where"}}], "locations": [{{"location": "exact", "context": "where"}}], "addresses": [{{"address": "exact", "associated_name": null, "context": "where"}}]}}

And its worked well so far. That being said, I bought a processing machine (the AMD StrixHalo 395+ AI Max), so I have a machine which can hold up to a 60gb model in memory. Once its done processing the OC, then I'm planning to start the next step in the process.

That being said, you are more than welcome to hop in on the repo I published and do whatever you please. Its all available to you.