56 scanned pages to a cited answer in 5 minutes: RAG pipeline for the documents search cannot read

This is an engineering tutorial, not a paper, but it lays out the path from scanned PDF to a cited answer clearly enough that teams building document question answering should read it. The key change: a pipeline that used to chain layout analysis, text recognition, table parsing, chunking and embedding now has two steps: one vision-language model reads each page, and one field type chunks and embeds at ingest. The demo goes from a 56-page scanned plant guide to a cited answer in under five minutes. I think it shows that the pipeline can be built; how well it works is for your own evaluation.

What problem does it solve?

Much knowledge lives in scanned PDFs, forms and photographed documents rather than databases. Classic OCR (optical character recognition) pipelines chain four models for layout detection, text recognition, table parsing and reading order, each with its own cost and failure modes, and tables often end up as unstructured lines.

The demo is a store selling Texas native plants. A shopper uploads a 56-page guide co-developed by the City of Austin and the Texas AgriLife Extension Service and asks which trees tolerate shallow limestone soil. The system answers from the guide, cites pages and recommends plants in stock. Almost every page is a dense multi-column table with abbreviations such as VL (very low water), check marks and italic botanical names.

Architecture: two paths

On upload, each PDF page is rendered to an image, sent to jina-ocr-v1 (Jina's document parsing model), and returned as structured Markdown that goes into Elasticsearch. On a question, hybrid retrieval finds passages and products, and Claude Haiku 4.5 writes a cited answer. All models are hosted on EIS (Elastic Inference Service), billed per token, with no GPU to deploy.

The author picks the third route. I think its real value is the security boundary: document content stays inside one project.

Step 1: scanned pages to clean Markdown

jina-ocr-v1 handles layout, text, tables and reading order in one pass. Its output keeps reading order and headings, emits tables as HTML with values intact, transcribes handwriting in place, and drops running headers, footers and printed page numbers. The program tracks page numbers itself.

The model reads one page image per call, legible up to about 2048 × 2048 pixels. Pages are rendered at about 1224 × 1584. Worked example (illustrative): that is a US letter page of 8.5 × 11 inches at 144 pixels per inch, about 1.94M pixels, under half of the 4.19M-pixel ceiling.

Four pages run in parallel and order is preserved. A rough bound (illustrative): 300 seconds × 4 workers ÷ 56 pages ≈ 21 seconds per page at most, including rendering, network and indexing.

Step 2: chunking and embedding at ingest

Each upload gets its own index named after the file. The content field is indexed as text and copied into a semantic_text field, which chunks the text at ingest, calls the project's default embedding endpoint, and embeds queries automatically. No chunking code, embedding job or vector bookkeeping is needed. A small registry index lists parsed documents for the chat.

Step 3: hybrid retrieval and cited answers

Each question runs two hybrid searches in parallel, over documents and over the catalog. Each uses RRF (Reciprocal Rank Fusion) over a keyword match, which catches exact names like Quercus macrocarpa, and a semantic query, which catches meaning that never appears verbatim.

RRF gives each document the sum of 1 ÷ (k + rank) across lists, with k = 60 by default. Worked example (illustrative): a document ranked 1st by keywords and 3rd semantically scores 1/61 + 1/63 ≈ 0.0323. A document ranked 1st only semantically scores 1/61 ≈ 0.0164. RRF uses ranks only, so score scales do not matter.

Page citations come from the open-source server/documents.js. Each page's Markdown is prefixed with a comment marker <!-- page N -->, and all pages are joined into one record. Search keeps the top 3 hits. Each hit gives up to 4 semantic fragments and up to 2 keyword fragments of about 350 characters. Hits are processed in rank order, semantic fragments first, near-duplicates are dropped, and only the first 5 fragments overall are kept. A regular expression reads the page marker inside each fragment. Worked example (illustrative): a fragment containing "<!-- page 50 --> invasive plants" is cited as page 50. Hoot says▾

Hoot saysHow the 5 fragment slots fill upClose ✕

Slots go first-come by hit rank: hit 1's 4 semantic fragments + 1 keyword fragment fill all 5, so the next two hits never get in, however relevant.

1. Slots: 4 + 1 = 5; hits 2 and 3 get 0 (illustrative)
2. If hit 1 had only 2 semantic fragments: 2 + 2 = 4, and hit 2 could add 1 (illustrative)

Drawn from the author's open-source server/documents.js; fragment counts are illustrative. Source: post + server/documents.js: top 3 hits, ≤ 4 semantic and ≤ 2 keyword fragments each, near-duplicates dropped, only the first 5 overall; pages come from <!-- page N --> markers in a fragment

So page numbers depend on luck. A fragment without a marker gets no page, and a fragment spanning two pages takes the first marker, which is the earlier page. The model is told to answer only from sources, cite name and page, recommend only catalog products, and say when the answer is missing. A hidden line of recommended product IDs is stripped from the stream, so a store with nothing suitable shows no product cards.

Limitations I see

1. The evidence is a demo. The author says hybrid retrieval consistently surfaced plants that keyword matching missed, shown in a compare view. Measure recall, citation accuracy and table accuracy on your own documents.
2. Page-level processing with coarse citations. Cross-page tables get split, and page numbers only appear when a fragment contains a marker. Storing one record per page or a page field per chunk would be sturdier.
3. One index per document will not scale to thousands of documents; one index with a document field is more likely in production.
4. Parsing errors are silent. A misread number in a table gets embedded, retrieved and cited with a correct page, which makes it look more trustworthy.

How a document QA team could try it

1. Pick 20 of your hardest scans and write 50 questions with answers and page labels.
2. Compare plain PDF text extraction, a classic OCR pipeline and a vision-language parser on table-cell accuracy and recall at 5.
3. On the same parsed text, compare keyword, semantic and RRF hybrid retrieval by question type.
4. Audit citations: is the page right, and does the cited text support the answer?

Verdict

The post is a reference implementation you can copy: a vision-language parser plus ingest-time embedding cuts scanned-document QA setup to an afternoon. Measuring quality is left to you.