{"record":{"id":"f4cf2a0a3af7d157","repo":"BerriAI/litellm","slug":"llm-as-a-judge-guardrail-requires-judge-model-in-l","errorCode":null,"errorMessage":"llm_as_a_judge guardrail requires judge_model in litellm_params","messagePattern":"llm_as_a_judge guardrail requires judge_model in litellm_params","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"litellm/proxy/guardrails/guardrail_hooks/llm_as_a_judge/__init__.py","lineNumber":243,"sourceCode":"                request_data=request_data,\n                guardrail_status=status,\n                start_time=start_time.timestamp(),\n                end_time=datetime.now().timestamp(),\n                event_type=GuardrailEventHooks.post_call,\n            )\n\n\ndef initialize_guardrail(\n    litellm_params: \"LitellmParams\",\n    guardrail: \"Guardrail\",\n) -> LLMAsAJudgeGuardrail:\n    guardrail_name: Final = guardrail.get(\"guardrail_name\")\n    if not guardrail_name:\n        raise ValueError(\"llm_as_a_judge guardrail requires a guardrail_name\")\n\n    judge_model: Final = _get_litellm_param(litellm_params, guardrail, \"judge_model\")\n    if not judge_model:\n        raise ValueError(\"llm_as_a_judge guardrail requires judge_model in litellm_params\")\n\n    criteria: Final = _get_litellm_param(litellm_params, guardrail, \"criteria\") or []\n    if not criteria:\n        raise ValueError(\"llm_as_a_judge guardrail requires at least one criterion\")\n\n    weight_total: Final = sum(float(c.get(\"weight\", 0)) for c in criteria)\n    if abs(weight_total - 100) > 0.5:\n        raise ValueError(f\"llm_as_a_judge criterion weights must sum to 100 (got {weight_total})\")\n\n    on_failure: Final = _get_litellm_param(litellm_params, guardrail, \"on_failure\", \"block\")\n    if on_failure not in _VALID_ON_FAILURE:\n        raise ValueError(f\"llm_as_a_judge on_failure must be 'block' or 'log', got '{on_failure}'\")\n\n    overall_threshold: Final = float(_get_litellm_param(litellm_params, guardrail, \"overall_threshold\", 80.0))\n\n    mode: Final = _get_litellm_param(litellm_params, guardrail, \"mode\")\n    event_hook: GuardrailEventHooks | None = None\n    if isinstance(mode, str) and mode in {e.value for e in GuardrailEventHooks}:","sourceCodeStart":225,"sourceCodeEnd":261,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/proxy/guardrails/guardrail_hooks/llm_as_a_judge/__init__.py#L225-L261","documentation":"llm_as_a_judge needs a model to evaluate responses; initialize_guardrail() resolves judge_model via _get_litellm_param from litellm_params (falling back to the guardrail entry) and raises ValueError when it is absent or empty. It fires at config load time, before any traffic is judged.","triggerScenarios":"A litellm_llm_as_a_judge guardrail entry whose litellm_params block has criteria but no judge_model key (or judge_model set to an empty string).","commonSituations":"Configs that specify criteria and threshold but assume the judge reuses the deployment's model; typos like judge-model or judgemodel; empty value left from templating.","solutions":["Add judge_model to litellm_params, e.g. judge_model: gpt-4o","Make sure the key is spelled exactly judge_model and the value is non-empty","Confirm the judge model is deployed/routable on the proxy so the guardrail can call it"],"exampleFix":"# before\n litellm_params:\n   criteria: [{name: grounded, weight: 100}]\n\n# after\n litellm_params:\n   judge_model: gpt-4o\n   criteria: [{name: grounded, weight: 100}]","handlingStrategy":"validation","validationCode":"REQUIRED_LLM_JUDGE_KEYS = {\"judge_model\", \"criteria\"}  \n  \ndef validate_llm_judge_params(litellm_params: dict) -> None:  \n    missing = REQUIRED_LLM_JUDGE_KEYS - set(litellm_params)  \n    assert not missing, f\"llm_as_a_judge litellm_params missing: {sorted(missing)}\"  \n    assert litellm_params[\"judge_model\"], \"judge_model must be non-empty\"","typeGuard":"def is_valid_judge_config(lp: object) -> bool:  \n    return (  \n        isinstance(lp, dict)  \n        and isinstance(lp.get(\"judge_model\"), str)  \n        and bool(lp[\"judge_model\"].strip())  \n        and isinstance(lp.get(\"criteria\"), list)  \n        and len(lp[\"criteria\"]) > 0  \n    )","tryCatchPattern":null,"preventionTips":["Template llm_as_a_judge entries from one canonical example that includes judge_model","Confirm the judge model exists as a deployment on the proxy so scoring calls can route"],"tags":["llm-as-a-judge","guardrail","config","startup"],"backgroundTag":"config-validation-failed","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","schemaVersion":2},"datasetVersion":"2026-08-21T18:17:14.833Z"}