TraceLlmProvider({}): turn {current} requested more LLM resp

Error message

TraceLlmProvider({}): turn {current} requested more LLM responses than the trace provides for that turn

What it means

`TraceLlmProvider::chat` pops the next scripted step for the current turn; when that turn's queue is empty it bails, naming the current turn index. The agent requested more LLM responses than the trace scripts for that turn — the fixture under-specifies the round-trips, often because the agent looped (e.g. retried after an unexpected tool result) and exhausted its scripted steps.

Source

Thrown at crates/zeroclaw-eval/src/replay.rs:109

        _model: &str,
        _temperature: Option<f64>,
    ) -> anyhow::Result<String> {
        // Not exercised by the agent loop (which uses `chat`); kept for trait completeness.
        Ok(String::new())
    }

    async fn chat(
        &self,
        _request: ChatRequest<'_>,
        _model: &str,
        _temperature: Option<f64>,
    ) -> anyhow::Result<ChatResponse> {
        let step = {
            let mut state = self.state.lock().unwrap();
            let current = state.current;
            match state.turns.get_mut(current).and_then(|q| q.pop_front()) {
                Some(step) => step,
                None => anyhow::bail!(
                    "TraceLlmProvider({}): turn {current} requested more LLM responses than the trace provides for that turn",
                    self.trace_name
                ),
            }
        };
        match step {
            TraceResponse::Text {
                content,
                input_tokens,
                output_tokens,
            } => Ok(ChatResponse {
                text: Some(content),
                tool_calls: vec![],
                usage: Some(TokenUsage {
                    input_tokens: Some(input_tokens),
                    output_tokens: Some(output_tokens),
                    cached_input_tokens: None,
                }),

View on GitHub (pinned to 88bb9c8533)

Solutions

Add the missing step(s) to the reported turn in the trace fixture so the queue covers every `chat()` call
If the extra call comes from an agent retry, fix the scripted tool output so the agent does not need to retry
Re-record the trace against the current agent behavior

Example fix

// before — turn 1 scripts 1 step but the agent calls chat() twice
"turns": [[{"text":"a"}],[{"text":"b"}]]
// after
"turns": [[{"text":"a"}],[{"text":"b"},{"text":"b-followup"}]]

Defensive patterns

Strategy: validation

Validate before calling

// Before running a case, assert every scripted tool call in the trace will
// actually be requested: each tool_call step must correspond to a tool the
// case's toolset exposes and the goal exercises.
fn trace_tools_available(trace: &Trace, tools: &[ToolSpec]) -> anyhow::Result<()> {
    for step in trace.turns.iter().flatten() {
        if let Some(name) = step.tool_name() {
            if !tools.iter().any(|t| t.name == name) {
                anyhow::bail!("trace references tool '{name}' not in the case toolset");
            }
        }
    }
    Ok(())
}

Try / catch

match provider.chat(...).await {
    Err(e) if e.to_string().contains("more LLM responses than the trace") => {
        // under-scripted turn: report as fixture mismatch with the turn index
        report.record_fixture_mismatch(case_name, e);
    }
    Err(e) => return Err(e),
    Ok(resp) => resp,
}

Prevention

Script deterministic tool outputs so the agent never needs an unplanned retry round-trip
After editing agent prompts or tool behavior, re-run and re-export the trace
Err on the side of extra terminal steps only when the agent flow guarantees them

When it happens

Trigger: An agent retry loop that makes one extra `chat()` call per turn; a trace turn array with too few entries; steps accidentally placed in the wrong turn index when splitting a recorded session.

Common situations: Hand-authoring traces with too few steps per turn; scripted tool outputs that do not match what the agent expects, forcing a clarifying round-trip; off-by-one errors at turn boundaries.

Related errors

AI-assisted analysis of zeroclaw-labs/zeroclaw@88bb9c8533 (2026-08-23). Data as JSON: /api/errors/506ad0f854f18bf3. Report an issue: GitHub.