DR Pixel: Page-Native Deep Research with Vision-Language Models

Letting a research agent read page screenshots, instead of first converting PDFs to text and searching that, raises question-answering accuracy and needs fewer searches. DR Pixel renders 99K papers into about 1.06 million page images and encodes each image as one vector. The agent issues text queries and gets back the 3 most relevant page images. It keeps looking at these original images for the whole trajectory, so tables and curves are never distorted by text conversion. With no training at all, just switching to pages, GPT-5.4, Gemini-3.5-Flash and Qwen3.6-27B all gain more than 5 points over converting PDFs to text plus a separate figure-search tool, while making 20% to 30% fewer retrieval calls.

The second contribution is a way to train an open model, called counterfactual credit reweighting. The training data are search-and-reasoning records left by the stronger GPT-5.4, called teacher trajectories. Before training, each step is scored with the training question's gold answer by how much it raises that answer's probability; fine-tuning skips useless steps and weights the rest by score. Qwen3.6-27B trained this way goes from 40.1 to 52.1 accuracy, matching GPT-5.4 at 51.9, and it finds the correct page more often. The paper also releases DRPixelBench, 500 questions, each tied to one gold page. People working on document retrieval, multimodal RAG or search agents should read it.

What problem it solves

A typical "deep research" agent searches over several rounds and answers from what it finds. When the corpus is papers, the usual approach converts PDFs to plain text, splits it into chunks, and runs text retrieval.

Conversion loses information. Table rows and columns get scrambled, values in plots cannot be read at all, and equations are often mangled. Some systems add a separate "figure search" tool, but agents rarely choose to use it. In the paper's experiments, adding the figure tool lifts accuracy only 1 to 3 points over text alone.

DR Pixel's idea is simple: the original page already contains everything, so skip the conversion and let a model that can see images read the pages directly.

How the system is built

Offline, each paper is rendered page by page. Each page image is encoded by Qwen3-VL-Embedding-8B into one fixed-length vector and stored in an index.

At inference time the agent loops through "think, search, read results". It writes a text query, the retriever encodes the query into the same vector space, and returns the 3 most similar page images. The agent is a vision-language model that reads these images directly, then decides what to search next or gives an answer. Each question allows at most 20 searches.

For fairness, the text retriever returns 5 chunks of 256 words each time. 3 pages converted to text come to about 1,359 words and 5 chunks to about 1,257 words, so both sides get about the same amount of text per search.

No training, just reading pages

On the paper's own DRPixelBench, the three models score as follows.

Broken down by evidence source, GPT-5.4 gains most on questions that need tables and figures. On questions grounded only in body text or equations, DR Pixel is on par with both text-conversion setups.

The open model lags well behind. Under DR Pixel, Qwen3.6-27B is 11.8 points below GPT-5.4. The paper splits the gap into two parts. First, finding: it retrieves the correct page 9.4 points less often, and 5.3% of its queries repeat an earlier one. Second, using: when the correct page came from the latest search, it trails by only 5.1 points; when the page was found in an earlier search, the gap widens to 12.3. This is the problem the training method below addresses.

Training: learn only the useful steps

Training data are GPT-5.4 trajectories on training questions. Each question is run 6 times, only runs with a correct final answer are kept, and at most the 3 shortest per question remain, about 2,500 in total. Training papers do not overlap with evaluation papers. The simplest approach is supervised fine-tuning on every trajectory with all steps treated equally. The problem is that teacher trajectories also contain useless searches, and the student learns those too.

The paper gives each step a "contribution score". All of this is done offline before training. Scoring needs the training question's gold answer; at inference there is no gold answer and this step does not run. The scoring model is a frozen copy of the untrained Qwen3.6-27B.

It works in three steps.

Step one defines a scoring function: given the trajectory up to some step, how high is the frozen base model's log-probability of generating the gold answer.

Step two computes contributions for two kinds of steps. For a search, compare the gold answer's score under two contexts: one ends at the pages returned by the previous search, and the other adds this step's reasoning, query and returned pages. How much higher the second is than the first is this search's contribution. For a piece of reasoning, the scored context is the whole trajectory up to the pages returned by the last search, without the teacher's final reasoning and its answer; the gold answer is only the target being scored after it. Compare this context with the same context minus only that piece of reasoning, keeping all other reasoning, queries and search results as is, and see how much the gold answer's score drops. Hoot says▾

Hoot saysHow one trajectory gets a contribution score for each step (illustrative)Close ✕

Searches compare before and after, reasoning deletes only itself; zero-weight steps get no loss but stay in context

‹1/3›

Swipe or use the arrows for the three steps. All scoring happens offline before training, using a frozen base model to compute the gold answer's log-probability; it never runs at inference time, and no gold answer is available then. All numbers are illustrative. Source: illustrative example of the paper's §3 credit scores (Figure 3)

Step three turns contributions into training weights. Steps with contribution at or below 0 start with weight 0. Steps with positive contribution are ranked from small to large among same-kind steps across all training data; with rank fraction k/n, the weight is twice that, so it averages about 1. The final answer always gets weight 1.5. Each weight multiplies the training loss on the tokens that step itself outputs: reasoning text for reasoning, the issued query for search. Steps with weight 0 simply contribute no loss and stay in the context.

Reasoning and search also correct each other. A piece of reasoning is kept only if it passes two checks. First, the search it leads to is not masked. Second, keeping the earlier trajectory unchanged and replacing only this reasoning with an empty block, the log-probability of the query that follows it drops, and the drop is at least the median across reasoning steps. Reasoning that passes both checks but scored at or below 0 gets the median weight of the positively weighted reasoning steps. In the end about 62.6% of reasoning steps and 24.5% of search steps are masked.

Results after training

Plain fine-tuning also improves accuracy, but uses 57% more searches than weighted fine-tuning (14.0 vs 8.9). It uses up all 20 searches on 35.8% of questions, versus 9.8% for weighted fine-tuning. This shows that weighting the steps mainly cuts redundant searches.

The weighted model finds the correct page 6.3 points more often than GPT-5.4, though it is still less accurate once the page is found. So most of the accuracy gain comes from finding better.

In the table, "Finds correct page" is the share of questions where the gold page was retrieved at some point, and "Correct once found" is accuracy on those questions. Both are counted after the run against annotations; the agent itself never knows which page is gold.

Training used only the paper corpus, yet it also adds 9.6 to 14.4 points on three other paper QA benchmarks, IRPAPERS, M3SciQA and AirQA. On ViDoRe-V3, the trained model gains about 4.1 points with either page or text retrieval, ending 1.8 points below an upper bound that is handed the gold pages. ViDoRe-V3 covers enterprise documents from 8 domains such as finance, pharma and HR, in English and French. Only 1.7% of its questions truly need visuals, which is why the two retrieval modes score about the same there.

Limitations as I see them

Benchmark questions are generated by a model from abstracts and then rewritten by humans. Answers are judged by GPT-5.4-mini with a majority of three votes. The teacher GPT-5.4 and the judge come from the same model family, so judging may favor the teacher's answer style.

Each page image costs more tokens than the same content as text. In Figure 5 the paper lowers page resolution, cutting tokens by 24% to 45% for a 1 to 4 point accuracy drop. Even at the lower resolution, because it searches less, total input is still 22% to 33% below the text-conversion setup. A deployment should measure this trade-off on its own documents.

How a retrieval team could try it

If your corpus is mostly PDFs, reports or manuals, first train nothing: just swap text retrieval for page-image retrieval, use an off-the-shelf vision-language model as the agent, and check whether questions involving tables and figures clearly improve.

If you train your own model, contribution weighting does not depend on the task. With gold answers for the training questions and teacher trajectories, a frozen model can score each step, and useless steps get weight 0.

Verdict

Both ideas are practical. The first, reading pages directly, gives a steady gain with no training and fewer searches. The second, fine-tuning weighted by step contribution, lets an open model match its teacher without the over-searching that plain fine-tuning causes. The main uncertainty is that both question generation and judging rely on large models.