{"record":{"id":"8c17afb4cd0811f7","repo":"langchain-ai/deepagents","slug":"classifier-result-did-not-contain-exactly-one-deci","errorCode":null,"errorMessage":"Classifier result did not contain exactly one decision per reviewed call","messagePattern":"Classifier result did not contain exactly one decision per reviewed call","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"libs/code/deepagents_code/auto_mode.py","lineNumber":2101,"sourceCode":"        if isinstance(value, str) and value:\n            return value\n    return type(model).__name__\n\n\ndef _validate_classifier_ids(batch: AutoDecisionBatch, expected_ids: set[str]) -> None:\n    \"\"\"Validate exact one-to-one classifier coverage.\n\n    Args:\n        batch: Structured classifier result.\n        expected_ids: Tool-call IDs requiring model review.\n\n    Raises:\n        ValueError: If IDs are missing, duplicated, or unknown.\n    \"\"\"\n    actual_ids = [decision.tool_call_id for decision in batch.decisions]\n    if len(actual_ids) != len(set(actual_ids)) or set(actual_ids) != expected_ids:\n        msg = \"Classifier result did not contain exactly one decision per reviewed call\"\n        raise ValueError(msg)\n\n\nclass AutoModeHITLMiddleware(HumanInTheLoopMiddleware[AutoModeState, Any, Any]):\n    \"\"\"Apply deterministic policy, classifier review, and HITL fallback.\"\"\"\n\n    trace_policy = TracePolicy(process_inputs=omit_payload)\n    \"\"\"Omit hook inputs from traces by default; set a `TracePolicy` to override.\"\"\"\n\n    state_schema = AutoModeState\n\n    @property\n    def name(self) -> str:\n        \"\"\"Replace the stock main-agent HITL middleware by name.\"\"\"\n        return \"HumanInTheLoopMiddleware\"\n\n    def __init__(\n        self,\n        interrupt_on: Mapping[str, bool | InterruptOnConfig],","sourceCodeStart":2083,"sourceCodeEnd":2119,"githubUrl":"https://github.com/langchain-ai/deepagents/blob/a1af029e6e73cb17c36bff823d227747b28e91e1/libs/code/deepagents_code/auto_mode.py#L2083-L2119","documentation":"When Auto Mode delegates review of a batch of gated tool calls to a classifier, the classifier must return exactly one decision per reviewed call, with no missing, duplicated, or unknown tool_call_ids. `_validate_classifier_ids` (called from `awrap_model_call`) enforces this by comparing the decision ids against the expected id set. Any mismatch means the classifier output is unusable, so the middleware fails fast with this ValueError.","triggerScenarios":"The classifier model returns a batch whose decisions contain: a tool_call_id not in the reviewed batch, a duplicated id (len(actual_ids) != len(set(actual_ids))), or omits an id present in the batch — typically because the classifier LLM truncated, hallucinated, or reformatted ids.","commonSituations":"A weak/small classifier model emitting free-text instead of the expected structured decisions; prompt changes that drop the tool_call_id field; long batches where the model truncates output; classifier response parsing that mangles ids.","solutions":["Use a structured-output-capable classifier model or stricter response schema so every reviewed tool_call_id gets exactly one decision.","Retry the classifier with a prompt that emphasizes returning one decision per call id, verbatim from the request.","For long batches, split review into smaller chunks so the classifier cannot truncate decisions.","Log the classifier's raw output to identify which ids were missing, duplicated, or unknown."],"exampleFix":"// before\nclassifier_response = '{\"decisions\": [{\"tool_call_id\": \"call_1\", \"decision\": \"approve\"}]}'  // call_2 missing\n// after\nclassifier_response = '{\"decisions\": [{\"tool_call_id\": \"call_1\", \"decision\": \"approve\"}, {\"tool_call_id\": \"call_2\", \"decision\": \"reject\"}]}'","handlingStrategy":"validation","validationCode":"expected = {c['id'] for c in batch.tool_calls}\nactual = [d.tool_call_id for d in batch.decisions]\nif len(actual) != len(set(actual)) or set(actual) != expected:\n    # retry the classifier before consuming results\n    batch = rerun_classifier(batch.tool_calls)","typeGuard":"def classifier_batch_is_complete(decisions: list[Decision], expected_ids: set[str]) -> bool:\n    ids = [d.tool_call_id for d in decisions]\n    return len(ids) == len(set(ids)) and set(ids) == expected_ids","tryCatchPattern":"try:\n    validated = middleware_validate(batch)\nexcept ValueError as e:\n    if 'exactly one decision per reviewed call' in str(e):\n        batch = retry_classifier_with_stricter_prompt(batch)  # or fall back to HITL review\n    else:\n        raise","preventionTips":["Use a model with structured-output support for classification and validate the response schema before the middleware does.","Keep review batches small enough that the classifier cannot truncate output.","Log raw classifier output so missing/duplicated/unknown ids can be diagnosed quickly.","Include verbatim tool_call_ids in the classifier prompt and forbid the model from inventing ids."],"tags":["classifier","validation","auto-mode","llm-output"],"backgroundTag":"classifier-decision-id-mismatch","analyzedSha":"a1af029e6e73cb17c36bff823d227747b28e91e1","analyzedAt":"2026-08-29T11:43:24.718Z","schemaVersion":2},"datasetVersion":"2026-08-29T12:17:43.993Z"}