{"record":{"id":"66be645c3d73d843","repo":"BerriAI/litellm","slug":"llm-as-a-judge-guardrail-requires-a-guardrail-name","errorCode":null,"errorMessage":"llm_as_a_judge guardrail requires a guardrail_name","messagePattern":"llm_as_a_judge guardrail requires a guardrail_name","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"litellm/proxy/guardrails/guardrail_hooks/llm_as_a_judge/__init__.py","lineNumber":239,"sourceCode":"        finally:\n            self.add_standard_logging_guardrail_information_to_request_data(\n                guardrail_provider=\"llm_as_a_judge\",\n                guardrail_json_response=judge_result,\n                request_data=request_data,\n                guardrail_status=status,\n                start_time=start_time.timestamp(),\n                end_time=datetime.now().timestamp(),\n                event_type=GuardrailEventHooks.post_call,\n            )\n\n\ndef initialize_guardrail(\n    litellm_params: \"LitellmParams\",\n    guardrail: \"Guardrail\",\n) -> LLMAsAJudgeGuardrail:\n    guardrail_name: Final = guardrail.get(\"guardrail_name\")\n    if not guardrail_name:\n        raise ValueError(\"llm_as_a_judge guardrail requires a guardrail_name\")\n\n    judge_model: Final = _get_litellm_param(litellm_params, guardrail, \"judge_model\")\n    if not judge_model:\n        raise ValueError(\"llm_as_a_judge guardrail requires judge_model in litellm_params\")\n\n    criteria: Final = _get_litellm_param(litellm_params, guardrail, \"criteria\") or []\n    if not criteria:\n        raise ValueError(\"llm_as_a_judge guardrail requires at least one criterion\")\n\n    weight_total: Final = sum(float(c.get(\"weight\", 0)) for c in criteria)\n    if abs(weight_total - 100) > 0.5:\n        raise ValueError(f\"llm_as_a_judge criterion weights must sum to 100 (got {weight_total})\")\n\n    on_failure: Final = _get_litellm_param(litellm_params, guardrail, \"on_failure\", \"block\")\n    if on_failure not in _VALID_ON_FAILURE:\n        raise ValueError(f\"llm_as_a_judge on_failure must be 'block' or 'log', got '{on_failure}'\")\n\n    overall_threshold: Final = float(_get_litellm_param(litellm_params, guardrail, \"overall_threshold\", 80.0))","sourceCodeStart":221,"sourceCodeEnd":257,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/proxy/guardrails/guardrail_hooks/llm_as_a_judge/__init__.py#L221-L257","documentation":"initialize_guardrail() for llm_as_a_judge reads guardrail_name from the guardrail entry and refuses to construct the guardrail without it. The ValueError surfaces at proxy startup or config reload, when a guardrails entry for litellm_llm_as_a_judge omits the guardrail_name field.","triggerScenarios":"A guardrails config entry with guardrail: litellm_llm_as_a_judge but no guardrail_name key at the entry's top level.","commonSituations":"Copy-pasted guardrail YAML where the name line got dropped; configs migrated from another format that treated guardrail_name as optional; entries built programmatically without the name.","solutions":["Add guardrail_name: <unique-name> to the guardrail entry in config.yaml","If generating config programmatically, assert the key exists before writing the guardrail entry"],"exampleFix":"# before\n guardrails:\n   - guardrail: litellm_llm_as_a_judge\n     litellm_params:\n       judge_model: gpt-4o\n\n# after\n guardrails:\n   - guardrail: litellm_llm_as_a_judge\n     guardrail_name: my-judge\n     litellm_params:\n       judge_model: gpt-4o","handlingStrategy":"validation","validationCode":"def validate_guardrail_config(guardrails: list[dict]) -> None:  \n    for g in guardrails:  \n        assert g.get(\"guardrail_name\"), f\"guardrail {g.get('guardrail')!r} is missing guardrail_name\"  \n  \nvalidate_guardrail_config(config[\"guardrails\"])  # run before proxy start / in CI","typeGuard":"def has_guardrail_name(entry: object) -> bool:  \n    return isinstance(entry, dict) and isinstance(entry.get(\"guardrail_name\"), str) and bool(entry[\"guardrail_name\"].strip())","tryCatchPattern":null,"preventionTips":["Add a CI config-lint that loads the YAML and asserts guardrail_name on every guardrails entry","Generate guardrail entries from typed dataclasses or pydantic models so a missing name fails locally"],"tags":["llm-as-a-judge","guardrail","config","startup"],"backgroundTag":"config-validation-failed","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","schemaVersion":2},"datasetVersion":"2026-08-21T18:17:14.833Z"}