RCRAG Converter

What is RAG, and why does it need your documents converted?

RAG - retrieval-augmented generation - makes your personal and work files AI readable by splitting them into passages, turning each passage into a vector, and looking up the relevant ones before the model answers.

The problem RAG solves

A language model knows what was in its training data and nothing else. Ask it about your company's refund policy and it will produce something plausible and wrong, because a fluent guess and a correct answer look identical coming out of a model. RAG fixes this by giving the model the actual passage to read before it answers.

Why documents have to be converted first

A model cannot search a folder. Retrieval works over vectors - lists of numbers that place similar meanings near each other - so every document has to be split into passages and each passage turned into one of those vectors. That conversion is the whole of what a RAG converter does, and doing it badly is why many RAG systems return confident nonsense.

Chunking decides more than the model does

The single largest quality lever is how a document is split. A chunk cut mid-sentence produces a vector describing a fragment whose subject sits in the previous chunk, so retrieval returns a passage that matched but cannot answer. Good chunking cuts on paragraph and sentence boundaries and overlaps consecutive chunks, so a fact that straddles a boundary still appears whole somewhere.

What you get out

The practical output is JSONL: one JSON object per line, each with an id, the passage text, its embedding and some metadata. There is no formal standard for RAG corpora - every vector database has its own binary format - but nearly all of them import newline-delimited JSON, which makes it the working standard.

More on converting for RAG

  • rusty_ragdocument ingest, chunking and retrieval
  • rusty_xmlWord, PowerPoint and Excel, without LibreOffice
  • rusty_jpeg / rusty_pngimage decode
  • rusty_flacaudio decode
  • rusty_zstdcompression
  • rusty_allocthe allocator, in wasm too
  • rusty_timeNTP timestamps on every chunk
  • spacedbthe storage seam, CRDT per entry
  • FFAIthe AI toolkit the three engines below come from
  • CarmentaOCR - reads scans and screenshots, in your browser
  • Mercuryspeech recognition - turns recordings into transcripts
  • Argusvision - describes images and video frames

MATA Corporate Holdings © 2026 — Made in Scottsdale, Arizona

Remade with Rust