Heading paths in RAG: filter and cite chunks by section
Heading paths record where a chunk sits in its document, such as Billing then Invoices, so a RAG system can filter retrieval to one section, cite the section as well as the file, and see why a wrong passage came back, provided the source has real headings that the converter can read.
How do I use document headings to improve RAG retrieval?
Keep the outline above each chunk as metadata, then use it three ways: filter a query to one section when the user names it, show the section title in the citation, and check it when an answer looks wrong. RAG Converter writes heading_path and section_title into each chunk's metadata when the source has real headings.
The list of headings above a chunk
A chunk is a few hundred words cut from a longer document. On its own it often loses the thing that made it make sense: which part of the document it came from. A paragraph that says "refunds are issued within 30 days" means something different under Returns than under Enterprise contracts. Anthropic made the same point in its Contextual Retrieval write-up: traditional RAG setups often lose context when a document is split, and adding context back reduced failed retrievals in their tests.
A heading path is the cheapest piece of that context. It is the list of headings above the chunk, from the top of the outline down. In a staff handbook, a chunk about invoice dates might sit under Billing, then Invoices. That list is short, exact and already written by the author of the document, so nothing has to be generated to get it.
What RAG Converter writes for each chunk
Each row of `chunks.jsonl` carries a `metadata` object. Alongside `doc_id`, `source`, `page`, reading-order `seq` and character offsets, it includes `heading_path`, an array of headings such as `["Billing","Invoices"]`, and `section_title`, the nearest heading, here `Invoices`. The JSONL guide walks through the rest of the record.
The chunker is built so those values stay honest. A heading closes the chunk before it, so one chunk does not straddle two sections, and overlap never crosses a section boundary. Word files are read as outlines, including the heading styles defined in `styles.xml`, and Markdown is read as the outline it already is. That is why the chunking strategies guide treats headings as natural boundaries.
The path is only as good as the source. A Word file where every title is bold body text, rather than a heading style, has no outline to read, and its chunks are likely to carry little or no path. Open one row of your own output and check before you build on it.
Filter retrieval to one section
When a user names a part of the document, such as "what does the security policy say about laptops", a plain vector search still scores every chunk in the corpus. A metadata filter narrows the candidates to chunks whose `heading_path` contains the named section, and the similarity search runs inside that set.
Most vector databases can do this with an array field. In Qdrant, a match any condition on an array payload is satisfied when at least one stored value matches, and the docs recommend a payload index on fields you filter. In Chroma, `$contains` checks whether an array metadata field includes a value, and `$in` matches a single field such as `section_title` against a list. Use filters when the section is explicit, and leave them off for open questions, or you will hide the answer that lives somewhere you did not expect. Hybrid search pairs well with this, because exact heading words are what keyword ranking is good at.
Cite the section, not just the file
A citation that says handbook.docx sends the reader to a forty-page file. A citation that says handbook.docx, Billing, Invoices sends them to the paragraph. Build the label from `source` plus the joined `heading_path`, and add `page` when the format has pages. The citations guide covers why the source name matters, and the section outline is the next level of detail.
Section labels also help the model. Putting the section title at the top of each retrieved passage in the prompt tells the model where a sentence came from, so it is less likely to mix a rule from one section with an exception from another.
Debug wrong answers by section
When an answer is wrong, the first question is which chunks came back. Log the `heading_path` of every retrieved chunk next to the question. A pattern usually shows up quickly: the right document but the wrong section, a table of contents ranking above the content, or a glossary that matches every query. Each one has a different fix, and you can only see it once the sections are visible.
The same log tells you when the outline itself is the problem. If many chunks share an empty path, the source had no real headings. Fixing the styles in the Word document and converting again is often quicker than tuning retrieval.
Try it on one structured document
Pick a document with a clean outline, such as a handbook or a policy, and convert it at the converter. Open `chunks.jsonl`, read the `heading_path` values, and add the field to your vector database as filterable metadata. The converter does not run queries for you; the persistent corpus and query API are still planned, so filtering happens in your own database. Browser conversion is a Preview capability, so check its current status on the site before relying on it for a production pipeline. In the free tier the conversion runs in your browser tab, within 8 MB per file and 400 chunks, and the file never leaves the device. The parsers come from Remade with Rust, the memory-safe libraries behind the converter.