BerriAI/litellm · error · ValueError

llm_as_a_judge criterion weights must sum to 100 (got {weigh

Error message

llm_as_a_judge criterion weights must sum to 100 (got {weight_total})

What it means

llm_as_a_judge computes a weighted overall score, so initialize_guardrail() enforces that criterion weights sum to 100 within a 0.5 tolerance and raises ValueError (echoing the actual total) otherwise. This guarantees threshold comparisons are on a 0-100 scale.

Source

Thrown at litellm/proxy/guardrails/guardrail_hooks/llm_as_a_judge/__init__.py:251

def initialize_guardrail(
    litellm_params: "LitellmParams",
    guardrail: "Guardrail",
) -> LLMAsAJudgeGuardrail:
    guardrail_name: Final = guardrail.get("guardrail_name")
    if not guardrail_name:
        raise ValueError("llm_as_a_judge guardrail requires a guardrail_name")

    judge_model: Final = _get_litellm_param(litellm_params, guardrail, "judge_model")
    if not judge_model:
        raise ValueError("llm_as_a_judge guardrail requires judge_model in litellm_params")

    criteria: Final = _get_litellm_param(litellm_params, guardrail, "criteria") or []
    if not criteria:
        raise ValueError("llm_as_a_judge guardrail requires at least one criterion")

    weight_total: Final = sum(float(c.get("weight", 0)) for c in criteria)
    if abs(weight_total - 100) > 0.5:
        raise ValueError(f"llm_as_a_judge criterion weights must sum to 100 (got {weight_total})")

    on_failure: Final = _get_litellm_param(litellm_params, guardrail, "on_failure", "block")
    if on_failure not in _VALID_ON_FAILURE:
        raise ValueError(f"llm_as_a_judge on_failure must be 'block' or 'log', got '{on_failure}'")

    overall_threshold: Final = float(_get_litellm_param(litellm_params, guardrail, "overall_threshold", 80.0))

    mode: Final = _get_litellm_param(litellm_params, guardrail, "mode")
    event_hook: GuardrailEventHooks | None = None
    if isinstance(mode, str) and mode in {e.value for e in GuardrailEventHooks}:
        event_hook = GuardrailEventHooks(mode)

    instance: Final = LLMAsAJudgeGuardrail(
        guardrail_name=guardrail_name,
        judge_model=judge_model,
        criteria=criteria,
        overall_threshold=overall_threshold,
        on_failure=on_failure,

View on GitHub (pinned to 77b7c6c40c)

Solutions

Rebalance the weights so they sum to exactly 100 (e.g. 50/30/20)
If a criterion is optional, give it weight 0 and redistribute the remainder
Give every criterion an explicit weight - omitted weights count as 0 and drag the total down

Example fix

# before - sums to 90
 criteria:
   - {name: grounded, weight: 30}
   - {name: concise, weight: 30}
   - {name: polite, weight: 30}

# after - sums to 100
 criteria:
   - {name: grounded, weight: 40}
   - {name: concise, weight: 30}
   - {name: polite, weight: 30}

Defensive patterns

Strategy: validation

Validate before calling

def validate_criteria_weights(criteria: list[dict]) -> None:  
    total = sum(float(c.get("weight", 0)) for c in criteria)  
    assert abs(total - 100) <= 0.5, f"criterion weights sum to {total}, must be 100 (+/- 0.5)"  
  
validate_criteria_weights(config_lp["criteria"])

Prevention

Give every criterion an explicit weight in config and sum-check them in CI
When adding or removing a criterion, rebalance all weights in the same change
Avoid equal splits over 3/6 criteria - rounding rarely lands exactly on 100

When it happens

Trigger: A criteria list whose weight fields sum to something other than 100 - e.g. three criteria at weight 30 each (total 90), or criteria copied from a config where one weight was edited without rebalancing the rest.

Common situations: Adding/removing a criterion without rebalancing weights; equal-split lists over 3 or 6 criteria (33.3... rounding); weights omitted (each defaults to 0) so the total is 0.

Understand the failure class

Background: Config validation failed: what "invalid value for {key}" and settings-rejection errors mean across 19 open-source libraries — this error's family across 19 libraries.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/accaaea44ae0aeb8. Report an issue: GitHub.