In multi-turn RAG dialogue, two cases need a clarifying question before answering: the question itself has an unclear referent—for example the assistant just recommended two credit cards and the user asks "what rewards does this card have"; or the question looks clear but retrieved documents support several answers—for example "is the language that evolved from Common Brittonic still spoken," with Welsh, Breton, and Cornish in the docs. IBM Research builds ClariEdge, a ~3,900-sample dataset that specially includes hard negatives that "look ambiguous but the referent is already fixed by context", then LoRA-tunes 3B-scale Granite models to decide whether to clarify and to list candidates when clarifying. Granite-4.1-3B LoRA reaches 96.6% judgment accuracy versus 79.4% for prompt-based GPT-4o and 73.1% for GPT-5.6 Sol; after training without hard negatives, hard-negative-class accuracy falls from 87.3% to 35.4% (Granite-4.0-micro LoRA).
1. When to clarify, and what to say
Existing datasets cover only part of this. CLAMBER evaluates single-sentence questions with no dialogue context and no retrieved docs; AmbigQA treats any question with multiple factual answers as ambiguous, but for "who controlled the Italian government in 1922" today's models answer "Mussolini and the Fascist Party" without clarifying. Real deployments need dialogue plus documents together.
Another hard case: many questions look vague but are already determined. If the user said "I work in the northeast branch" and then asks "when does this branch's service open," "this branch" is clear. Clarifying those over-interrupts the user.
2. Method
2.1 ClariEdge dataset
Dialogues cover MTRAG's four domains: ClapNQ (Wikipedia), Cloud (tech docs), FiQA (finance), Govt (government). Data fall into three classes.
Hard negatives are built by having Mistral-Large-3 ask about entity types that appear with multiple instances in 793 dialogues, plus step-by-step self-verified generation on 228 longer dialogues (>6 turns, many entity types). Humans then audit: truly ambiguous → Ambiguous; determined by context → CLEAR-hard. The authors also add patterns models often misjudge, such as "which battery lasts longest" (comparing all candidates, not pointing at one) and short follow-ups like "price?". Hard negatives appear only in the no-retrieval setting.
Each ambiguous sample gets a reference clarify, written in three styles by candidate count: list each with a short answer for 2–3 candidates; list candidates only for 4–8; ask which kind when candidates are many or unenumerable. Reference clarifies are Mistral-Large-3 drafts then human-edited. Two authors independently label 80 test samples for ambiguity; Cohen's Kappa is 0.875, with all 4 disagreements in CLEAR-hard.
2.2 Model and usage
LoRA-tune Granite-4.0-micro, Granite-4.1-3B, and GPT-OSS-20B on 3,509 training samples. At deployment the same LoRA is called twice: before retrieval, dialogue only, decide whether to clarify first; after retrieval, dialogue plus docs, decide whether to clarify before answering. Clarify if either call says ambiguous.
3. Experiments
Setup. Compare seven prompt-only models: GPT-5.6 Sol, GPT-4o, GPT-OSS-120B, GPT-OSS-20B, Granite-4.1-30B, Granite-4.1-3B, Granite-4.0-micro, all with the same system prompt and inputs. Evaluate on the full test set, pre-retrieval subset, and post-retrieval subset.
3.1 Whether to clarify
This checks whether a fine-tuned small model judges clarification need more accurately than larger prompt-only models.
(%,) Prompt models split into two error modes: GPT-5.6 Sol and GPT-OSS-20B tend to clarify everything (high ambiguous-class accuracy, hard-negative class only 48.1% and 58.2%); GPT-4o and small Granite tend not to clarify (low ambiguous-class accuracy). Same Granite-4.1-3B: prompt 64.8% versus LoRA 96.6%—structure identical, difference from training. Pre-retrieval is harder; LoRA's ambiguous-class accuracy there (84.5%) is below post-retrieval (100.0%).
3.2 Whether the clarify lists the right candidates
This checks whether listed candidates are correct and complete, only on samples the model correctly judged ambiguous. Precision is the share of listed candidates that are correct; recall is coverage of reference candidates; judged by four LLM prompts.
(%,) Prompt models have similar precision but low recall—listed candidates are mostly right but often miss equally valid options. On the pre-retrieval subset, GPT-5.6 Sol (85.1), GPT-OSS-20B (87.5), and Granite-4.1-30B (86.2) F1 match the two LoRAs (~85%), but their judgment accuracy is lower.
3.3 Role of hard negatives
This checks whether hard negatives are necessary: train Granite-4.0-micro LoRA the same way except drop all CLEAR-hard samples.
Without hard negatives the model clarifies almost every surface-ambiguous question; CLEAR-hard accuracy falls to 35.4%.
3.4 New domains
On MTRAG-UN banking and telecom, Granite-4.0-micro LoRA judgment accuracy is 92.6% and 90.2% versus at most 71.9% for prompt models (GPT-OSS-120B, telecom). On banking clarify-quality F1, GPT-4o (89.82%) and Granite-4.1-30B (86.61%) beat LoRA (81.56%), but they correctly judge fewer ambiguous samples (42/79 and 60/79 versus LoRA 69/79).
4. Limitations
The model emits one clarify and does not support multi-turn gradual clarification; data and eval are English only—both stated by the authors. Clarify-quality precision/recall rely on LLM judges; hard negatives cover only pre-retrieval, not post-retrieval "several answers look possible but can be merged"; the test set has only 104 ambiguous samples, so a few-point gap between models should not be over-read.
5. Takeaways
For multi-turn RAG assistants, "whether to clarify" can be a 3B-scale LoRA called once before and once after retrieval, without relying on a large prompt-only model. The key is training data that includes "looks ambiguous but context already fixed" counterexamples—otherwise the model clarifies too often. ClariEdge is public on Hugging Face (ibm-research/ClariEdge).