RAG with frontier models is a waste of money and a privacy violation
Sending your corpus through a frontier model to 'do RAG' wastes money on work a tiny embedding model can finish once, and hands contracts, medical notes and board papers to a third party when the bytes never needed to leave your machine.
Two jobs got collapsed into one bill
RAG has two stages that people keep buying as one product. Conversion turns documents into passages and vectors so they can be searched. Generation puts a handful of those passages in front of a language model so it can answer. The second job is where a frontier model earns its keep. The first job is translation into a searchable form - parsing, chunking, embedding - and it does not need a model that can write a novel or pass a bar exam. When a team 'does RAG with Claude' or 'indexes with GPT', they are usually paying frontier prices for the translation step, every page, every rebuild, while the privacy cost of that upload is treated as a footnote.
Translation does not need a frontier brain
Embedding a passage is a narrow task: map text into a vector so similar meanings sit near each other. A sentence transformer such as all-MiniLM-L6-v2 does that in a few hundred dimensions on a laptop, or in a browser tab as WebAssembly, for essentially free after the first download. Frontier models are priced for reasoning and long-form generation. Using them to restate, summarise or 'understand' every page before retrieval is like hiring a surgeon to sort mail. You get a plausible-looking corpus and a bill that scales with pages rather than with questions - the opposite of why RAG was invented. Convert once with a small model. Retrieve cheaply. Spend the frontier budget only on the answer.
The privacy compromise is not abstract
The documents people want searchable are the ones they cannot show a vendor: employment contracts, patient notes, diligence folders, source code, board packs, customer tickets with names and amounts. Uploading that material to a frontier provider so it can be 'RAG-ready' means a third party receives a copy - retained under their terms, reachable by subpoena, exposed if their systems are breached, and subject to whatever training or logging policy they publish next quarter. A privacy policy is a promise about intent. Local conversion is a change in what is possible: if the file never leaves the tab, there is no remote copy to argue about. Calling that risk 'accepted' because the product is convenient is how organisations sleepwalk into a disclosure they cannot undo.
What the bill is actually buying
Frontier conversion charges you for tokens on every page you already own. Rebuild the index after a folder changes and you pay again. Ask three questions about the same filing and a naive 'send the PDF to the model' design pays for the pages three times. A proper converter pays for understanding once - parse, chunk, embed - then every later question is a vector lookup plus a short prompt. That is the economics RAG was sold on. Using the most expensive model in the stack for the cheapest work destroys the bargain and still leaves you without a reusable corpus you control. The waste is not theoretical; it shows up as invoices that track page count instead of answer quality.
Where a frontier model still belongs
After retrieval. Give it the five passages that matched the question and ask it to answer only from those. That is generation with grounding, not conversion with a megaphone. Vision-capable frontier models can also help on a one-off page you will never index - a short PDF read once in chat - but that demo is not a system. Folders, second questions, citations and anything past a context window still need passages you own. Keep the frontier model on the answer path. Keep the corpus on your machine or on infrastructure you chose, produced by a small embedding model that never needed to see your secrets in someone else's datacenter.
A sane default
Convert locally by default: text-layer documents in the browser, optional on-device embeddings, JSONL you can load into any vector store. Use a paid conversion API only for the jobs that need horsepower you will not run in a tab - full-resolution OCR, speech, video - with eyes open about what leaves the building. Reserve GPT, Claude and their peers for reading the retrieved passages, not for translating the library. Anything else is overpaying for work a tiny model finishes once, and shipping private material to a place it did not need to go.