{"record":{"id":"9234c8a691e40355","repo":"BerriAI/litellm","slug":"baseline-model-is-required-when-direction-is-reve","errorCode":null,"errorMessage":"baseline_model is required when direction is 'reverse'","messagePattern":"baseline_model is required when direction is 'reverse'","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"litellm/types/management_endpoints/auto_router_endpoints.py","lineNumber":217,"sourceCode":"    max_turns: int = Field(\n        default=200,\n        ge=1,\n        le=2000,\n        description=(\n            \"Sample budget: the job judges at most this many turns, then completes. This is also the spend \"\n            \"bound; expected judge cost is roughly max_turns times one judge call\"\n        ),\n    )\n\n    @field_validator(\"shadow_percentage\")\n    @classmethod\n    def _round_percentage(cls, value: float) -> float:\n        return round(value, 2)\n\n    @model_validator(mode=\"after\")\n    def _baseline_model_matches_direction(self) -> \"StartShadowEvalRequest\":\n        if self.direction == \"reverse\" and self.baseline_model is None:\n            raise ValueError(\"baseline_model is required when direction is 'reverse'\")\n        if self.direction == \"forward\" and self.baseline_model is not None:\n            raise ValueError(\"baseline_model is only meaningful when direction is 'reverse'\")\n        return self\n\n\nclass ShadowEvalSlice(BaseModel):\n    \"\"\"Judge outcomes for one slice of a job's verdicts (a router tier, or one of the\n    models that served the real arm).\"\"\"\n\n    group: str\n    turn_count: int\n    real_win_rate_pct: float = Field(\n        description=(\n            \"Share of judged turns the real arm won, meaning the response the caller actually received: \"\n            \"the key's own model in forward mode, the router's pick in reverse\"\n        )\n    )\n    shadow_win_rate_pct: float = Field(","sourceCodeStart":199,"sourceCodeEnd":235,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/types/management_endpoints/auto_router_endpoints.py#L199-L235","documentation":"Cross-field model validator on StartShadowEvalRequest: when direction == 'reverse', a baseline_model must be supplied to compare against, and its absence makes the shadow-eval job unrunnable, so validation fails at request parse time.","triggerScenarios":"Thrown at litellm/types/management_endpoints/auto_router_endpoints.py:217 when the library encounters an invalid state.","commonSituations":"See trigger scenarios.","solutions":["Set baseline_model when using direction='reverse' so the comparison has a reference model.","Or use direction='forward' if no baseline comparison is wanted."],"exampleFix":null,"handlingStrategy":"validation","validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}