{"record":{"id":"5d344b5e6c86bb01","repo":"BerriAI/litellm","slug":"baseline-model-is-only-meaningful-when-direction-i","errorCode":null,"errorMessage":"baseline_model is only meaningful when direction is 'reverse'","messagePattern":"baseline_model is only meaningful when direction is 'reverse'","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"litellm/types/management_endpoints/auto_router_endpoints.py","lineNumber":219,"sourceCode":"        ge=1,\n        le=2000,\n        description=(\n            \"Sample budget: the job judges at most this many turns, then completes. This is also the spend \"\n            \"bound; expected judge cost is roughly max_turns times one judge call\"\n        ),\n    )\n\n    @field_validator(\"shadow_percentage\")\n    @classmethod\n    def _round_percentage(cls, value: float) -> float:\n        return round(value, 2)\n\n    @model_validator(mode=\"after\")\n    def _baseline_model_matches_direction(self) -> \"StartShadowEvalRequest\":\n        if self.direction == \"reverse\" and self.baseline_model is None:\n            raise ValueError(\"baseline_model is required when direction is 'reverse'\")\n        if self.direction == \"forward\" and self.baseline_model is not None:\n            raise ValueError(\"baseline_model is only meaningful when direction is 'reverse'\")\n        return self\n\n\nclass ShadowEvalSlice(BaseModel):\n    \"\"\"Judge outcomes for one slice of a job's verdicts (a router tier, or one of the\n    models that served the real arm).\"\"\"\n\n    group: str\n    turn_count: int\n    real_win_rate_pct: float = Field(\n        description=(\n            \"Share of judged turns the real arm won, meaning the response the caller actually received: \"\n            \"the key's own model in forward mode, the router's pick in reverse\"\n        )\n    )\n    shadow_win_rate_pct: float = Field(\n        description=(\n            \"Share of judged turns the shadow arm won, meaning the duplicated response nobody was served: \"","sourceCodeStart":201,"sourceCodeEnd":237,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/types/management_endpoints/auto_router_endpoints.py#L201-L237","documentation":"Pydantic model_validator on StartShadowEvalRequest: baseline_model was supplied while direction is not 'reverse'; in forward mode the baseline is meaningless, so the combination is rejected as likely misconfiguration.","triggerScenarios":"Thrown at litellm/types/management_endpoints/auto_router_endpoints.py:219 when the library encounters an invalid state.","commonSituations":"See trigger scenarios.","solutions":["Remove baseline_model when direction is not 'reverse'.","Or set direction='reverse' if a baseline comparison is intended."],"exampleFix":null,"handlingStrategy":"validation","validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}