Illustration of a stack of notebooks under a magnifying lens beam with one glowing page floating through a doorway toward a cloud

I have somewhere around 7,000 pages of notes. Troubleshooting tricks from my own homelab, self-study notes, weird fixes I figured out at 2 AM and wrote down so Future Me wouldn't have to suffer through them twice. Years of tinkering and late-night rabbit holes, distilled into what is — if I say so myself — a genuinely good personal knowledge base.

And it's nearly useless, because folders and Ctrl+F don't scale. What I actually want is to ask my notes questions. "How did I fix that weird DNS rabbit hole on my home network?" — and get my own answer back, in my own words, with the doc it came from.

That's RAG — retrieval-augmented generation. And this is the story of how I tried to build it the "right" way, faceplanted, and ended up with an architecture I like much better: one where everything lives on my own hardware except for a single hour-long visit to the cloud.

Attempt one: full self-host, faceplant

My first instinct was the purist one: run everything locally, let no document leave the building. I stood up RAGFlow — the open-source RAG engine with the built-in document parser — pointed it at my KB, and watched it choke.

The problem is that my notes are messy. Not "slightly untidy" messy — overlapping screenshots, pasted terminal output, tables inside tables, half-markdown half-chaos. Good OCR and layout understanding need proper vision models, and the small models you can realistically run on a laptop just aren't there yet. Retrieval quality was bad enough that the whole thing was pointless. Garbage in, garbage out, exactly as advertised.

The managed-RAG trap

So I looked at managed services — the "upload your docs, we handle everything" RAG platforms. And I noticed they all share a design flaw that very much isn't a bug: they're roach motels. Documents go in, answers come out, but you can never export the chunks or the embeddings. The lock-in is the point. I wanted preprocessing-as-a-service — upload, get my artifacts back, leave — and that's simply not what those products sell.

But the thing I wanted does exist. It's just a different product category: document-parsing APIs. Unstructured, LlamaParse, plus newer ones like Chunkr and Reducto. You send them ugly documents; they send back clean, structured, chunked JSON — tables extracted, screenshots OCR'd by actual vision models, metadata attached. Precisely the one step my laptop was failing at.

The economics nobody believes

Here's the part that broke my brain a little, in a good way. I'd assumed RAG had an expensive "heavy machinery" phase that justified outsourcing chunks of it. It doesn't. Once parsing is solved:

  • Chunking is string splitting with rules. A laptop chews through thousands of pages in minutes. No GPU, just code.
  • Embedding — turning chunks into vectors — is the only compute-ish step, and OpenAI's text-embedding-3-small costs $0.02 per million tokens. My entire KB is maybe five million tokens. That's a dime. Even on a fancier embedding model it's well under a dollar. Or run bge-m3 locally on CPU and pay nothing but fan noise.

The expensive GPU stuff only starts if you're training a model — and training is a separate project entirely. Fine-tuning changes a model's behavior; RAG gives it memory. I want memory. Memory turns out to be embarrassingly cheap.

But is my data safe with a parsing API? I read the ToS so you don't have to

My threat model is simple: this is my knowledge, my research, written on my own time on my own hardware — and I don't want it training someone else's model or lingering on someone's server. So before uploading anything, I did the thing nobody does: I read Unstructured's entire platform terms of service, front to back. (Their Transform free tier is 15,000 pages, by the way — my whole KB fits in one free run. LlamaParse's free tier is roughly ten thousand credits a month, also workable.)

The verdict, in plain language:

  • You own your data. They get a license to process it for you, nothing more.
  • No training. They contractually promise not to train models on your documents, and they've obtained the same promise from the third-party AI providers they route through. This is the clause that matters most, and it's solid.
  • Transient vs. persistent products. Their Transform and Pipelines products process and forget; their Foundation product keeps a persistent index. So: use the transient one, download your chunks, delete the account. They complete deletion within 30 days and will certify it in writing if you ask.
  • One landmine: a "Restricted Data" clause forbidding credentials, government IDs, health info, and anything else regulated. Translation: scrub your docs first. Personal notes always have that one page with a password or an API key buried in a procedure, and I'd bet money mine is no exception. That scrub pass is now step zero.
  • The fine print: total liability is capped at the greater of twelve months' fees or a hundred bucks. So "protected" means "they promise," not "they're on the hook." For personal notes, acceptable. For anything genuinely sensitive, that's the scenario where full self-hosting is the right answer instead.

I'm a tinkerer, not a lawyer — but as self-serve SaaS terms go, that's about as good as it gets.

The architecture, final form

So here's the pipeline I've landed on:

  1. Parse in the cloud. Upload docs to a parsing API. I'll test both Unstructured and LlamaParse against my single worst document first — the one with overlapping screenshots and janky formatting — and judge them on that, not the clean pages. Transient processing, real vision models doing the OCR.
  2. Download everything, delete everything. Get the chunked JSON out, then wipe the account. Total exposure window: about an hour. After that, the only copies on Earth are mine.
  3. Embed once. One API call costing dimes — or bge-m3 locally for zero third parties from that point on.
  4. Local vector DB. sqlite-vec or Chroma: a single file on disk, no server to babysit. Every chunk keeps its source metadata, so answers can cite the doc they came from — an assistant that can say "per your Proxmox rebuild notes" is trustworthy; one that answers from the void is not.
  5. Tiny MCP server. Fifty-ish lines wrapping the database in a single kb_search(query) tool, so any AI agent that speaks the Model Context Protocol can query my notes on demand — and return "nothing found" instead of hallucinating in my KB's voice. Same pattern I used when I built an MCP server for this blog. It works beautifully.

Everything after step one lives on my hardware, forever, no subscriptions. Total cost: probably $0, thanks to free tiers. Total infrastructure: a folder, a SQLite file, and a small script.

The takeaway

The heavy-machinery phase of RAG is a myth that survives because managed-RAG vendors profit from you believing it. Once embedding APIs exist, the only genuinely hard problem is parsing ugly documents — and you can rent exactly that capability for the price of a coffee, without surrendering custody of your knowledge.

So if you've got a pile of notes and a search problem, don't start by shopping for an all-in-one RAG platform. Start by asking which single step your hardware actually can't do — and outsource only that.

How much of your own knowledge is sitting in a folder somewhere, waiting to become something you can actually talk to?