RAG for tables: why flattening a grid ruins retrieval
A table only works in RAG if its row-column relationships survive conversion - flatten it into a run of numbers and retrieval can still match the values, but the model can no longer tell which heading any of them belonged to.
The data is the document
Earnings releases, 10-Qs, rate cards and technical briefs are not prose with a table attached. The table is the content. Hosted parsers - Unstructured, LlamaParse, Reducto - all treat this as their headline problem, because a standard chunk-and-embed job destroys exactly the structure those files exist to carry. If your questions are about figures in a grid, this is the conversion quality that matters, not overall text accuracy.
What flattening actually loses
A PDF table is position: cells sit under headers, sometimes under two header rows, sometimes spanning a page break. Dump the reading order into a paragraph and you get 'EMEA 412 2019 0.83 APAC 188'. The numbers are still in the corpus. The relationships are not. Retrieval then returns a passage that contains the right token and a model that cannot reconstruct which column it sat in, which is how a fluent answer cites the wrong year.
Keep the table, embed a sentence about it
The pattern the production parsers converged on is the same one you should use by hand: preserve the table as a structured object - HTML, Markdown pipes or JSON rows - and attach a short prose summary that names what the grid is. Embed the summary so retrieval can find it in ordinary language; keep the structured table on the chunk so the model can read the actual cells. Embedding the raw grid alone matches poorly; dropping the grid and keeping only the summary leaves you nothing to cite.
Do not split a table mid-row
A chunker that cuts on a character count will bisect a table the way it bisects a paragraph, and a half-row is worse than a missing one. Treat a table that fits as one chunk. If it does not fit, split on rows and repeat the headers on every piece, otherwise the second half is a list of numbers with no names. Unstructured's chunker does this as a rule; a homemade splitter will not unless you tell it a table is not prose.
Test tables on their own, not as part of a score
An overall parsing accuracy number hides this completely. A converter can score well on narrative pages and still flatten every grid. Take three of your own table-heavy files, convert them, and ask two questions that only the table can answer - a specific cell, and a comparison across a row. If the retrieved passage does not still look like a table, the rest of the pipeline cannot save it. RAG Converter reports per-file chunk counts in the manifest so an empty or tiny yield on a spreadsheet is visible instead of silent.