P4 — When is "done" unjustified?
User journey: knowns/unknowns ledger → agent CoT → reviewer veto → Crux MCQ → distill.
A-HA: the memo looked finished; the reasoning trace revealed a silent assume.
title: "Elicitation-Gated Agent Scaffolding with a Reasoning-Trace Reviewer" authors: Complide Research affiliations: OutdoorAGI Inc. (DBA Complide) date: 2026-08-03 tags: [elicitation, reviewer, distillation, uncertainty, agents] provisional: COMPLIDE-P4-PROV space: Complide/elicitation-reviewer
Abstract
Agents often close tasks with unjustified confidence. We scaffold elicitation with an uncertainty ledger (knowns/unknowns across interaction, content, and style axes), a separation-of-duties reviewer that reads the reasoning trace and vetoes premature closure with a targeted gap question, and a value-of- information gate that bounds over-asking. Scaffolded trajectories distill into models via SFT, preference optimization on veto pairs, or RL with the reviewer as a process-reward model. A white-box ask/ambiguity direction provides a training-free analog and runtime monitor.
1. Introduction
Conversation-only review misses silent assumptions buried in CoT. Reading the reasoning stream lets a reviewer catch premature closure before the user sees a confident wrong answer.
2. Method
- Uncertainty ledger. Enumerate knowns/unknowns; inject as context.
- Reviewer veto. Detect premature closure; inject a high-VOI gap question.
- VOI gate. Escalate while unknowns exceed a VOI threshold; trade coverage against turns.
- Distillation factory. Export
ask_sft/commit_prefdatasets; optional RL with reward = elicitation − (\lambda\cdot)turns. - White-box analog. Probe/steer an ask-or-ambiguity direction when weights are available.
3. Product surface
complide-reviewer agent and assess/ask CLI paths; RL-env / ReqElicitGym
harnesses on the research lineage. Shipping recommendation: runtime elicitation
plus customer-facing dataset emission (productization D8 option b).
4. Provisional notice
US provisional COMPLIDE-P4-PROV (Elicitation-Gated Agent Scaffolding with a Reasoning-Trace Reviewer and Distillation of Elicitation Behavior).
References
- ReqElicitGym / Complide reviewer agents; Complide umbrella paper.