{"record":{"id":"accaaea44ae0aeb8","repo":"BerriAI/litellm","slug":"llm-as-a-judge-criterion-weights-must-sum-to-100","errorCode":null,"errorMessage":"llm_as_a_judge criterion weights must sum to 100 (got {weight_total})","messagePattern":"llm_as_a_judge criterion weights must sum to 100 \\(got (.+?)\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"litellm/proxy/guardrails/guardrail_hooks/llm_as_a_judge/__init__.py","lineNumber":251,"sourceCode":"def initialize_guardrail(\n    litellm_params: \"LitellmParams\",\n    guardrail: \"Guardrail\",\n) -> LLMAsAJudgeGuardrail:\n    guardrail_name: Final = guardrail.get(\"guardrail_name\")\n    if not guardrail_name:\n        raise ValueError(\"llm_as_a_judge guardrail requires a guardrail_name\")\n\n    judge_model: Final = _get_litellm_param(litellm_params, guardrail, \"judge_model\")\n    if not judge_model:\n        raise ValueError(\"llm_as_a_judge guardrail requires judge_model in litellm_params\")\n\n    criteria: Final = _get_litellm_param(litellm_params, guardrail, \"criteria\") or []\n    if not criteria:\n        raise ValueError(\"llm_as_a_judge guardrail requires at least one criterion\")\n\n    weight_total: Final = sum(float(c.get(\"weight\", 0)) for c in criteria)\n    if abs(weight_total - 100) > 0.5:\n        raise ValueError(f\"llm_as_a_judge criterion weights must sum to 100 (got {weight_total})\")\n\n    on_failure: Final = _get_litellm_param(litellm_params, guardrail, \"on_failure\", \"block\")\n    if on_failure not in _VALID_ON_FAILURE:\n        raise ValueError(f\"llm_as_a_judge on_failure must be 'block' or 'log', got '{on_failure}'\")\n\n    overall_threshold: Final = float(_get_litellm_param(litellm_params, guardrail, \"overall_threshold\", 80.0))\n\n    mode: Final = _get_litellm_param(litellm_params, guardrail, \"mode\")\n    event_hook: GuardrailEventHooks | None = None\n    if isinstance(mode, str) and mode in {e.value for e in GuardrailEventHooks}:\n        event_hook = GuardrailEventHooks(mode)\n\n    instance: Final = LLMAsAJudgeGuardrail(\n        guardrail_name=guardrail_name,\n        judge_model=judge_model,\n        criteria=criteria,\n        overall_threshold=overall_threshold,\n        on_failure=on_failure,","sourceCodeStart":233,"sourceCodeEnd":269,"githubUrl":"https://github.com/BerriAI/litellm/blob/77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8/litellm/proxy/guardrails/guardrail_hooks/llm_as_a_judge/__init__.py#L233-L269","documentation":"llm_as_a_judge computes a weighted overall score, so initialize_guardrail() enforces that criterion weights sum to 100 within a 0.5 tolerance and raises ValueError (echoing the actual total) otherwise. This guarantees threshold comparisons are on a 0-100 scale.","triggerScenarios":"A criteria list whose weight fields sum to something other than 100 - e.g. three criteria at weight 30 each (total 90), or criteria copied from a config where one weight was edited without rebalancing the rest.","commonSituations":"Adding/removing a criterion without rebalancing weights; equal-split lists over 3 or 6 criteria (33.3... rounding); weights omitted (each defaults to 0) so the total is 0.","solutions":["Rebalance the weights so they sum to exactly 100 (e.g. 50/30/20)","If a criterion is optional, give it weight 0 and redistribute the remainder","Give every criterion an explicit weight - omitted weights count as 0 and drag the total down"],"exampleFix":"# before - sums to 90\n criteria:\n   - {name: grounded, weight: 30}\n   - {name: concise, weight: 30}\n   - {name: polite, weight: 30}\n\n# after - sums to 100\n criteria:\n   - {name: grounded, weight: 40}\n   - {name: concise, weight: 30}\n   - {name: polite, weight: 30}","handlingStrategy":"validation","validationCode":"def validate_criteria_weights(criteria: list[dict]) -> None:  \n    total = sum(float(c.get(\"weight\", 0)) for c in criteria)  \n    assert abs(total - 100) <= 0.5, f\"criterion weights sum to {total}, must be 100 (+/- 0.5)\"  \n  \nvalidate_criteria_weights(config_lp[\"criteria\"])","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Give every criterion an explicit weight in config and sum-check them in CI","When adding or removing a criterion, rebalance all weights in the same change","Avoid equal splits over 3/6 criteria - rounding rarely lands exactly on 100"],"tags":["llm-as-a-judge","guardrail","config","weights","startup"],"backgroundTag":"config-validation-failed","analyzedSha":"77b7c6c40c0c5aa5fbcb1d6a1825ac39ca8829b8","analyzedAt":"2026-08-18T11:44:31.656Z","schemaVersion":2},"datasetVersion":"2026-08-21T18:17:14.833Z"}