What causes "LLM judge rejected response: score below threshold" in BerriAI/litellm?

Lower overall_threshold in litellm_params to a value the judged model can realistically meet

BerriAI/litellm · error · HTTPException

LLM judge rejected response: score below threshold

Error message

LLM judge rejected response: score below threshold

What it means

The llm_as_a_judge guardrail scores each LLM response in a post_call hook against weighted criteria. When the weighted overall_score falls below overall_threshold (default 80) and on_failure is 'block' (the default), the proxy raises HTTPException 422 with the score, threshold, and per-criterion verdicts. With on_failure='log' the failure is recorded under metadata.eval_information and the response passes through.

Source

Thrown at litellm/proxy/guardrails/guardrail_hooks/llm_as_a_judge/__init__.py:204

                "overall_score": overall_score,
                "passed": passed,
                "judge_model": self.judge_model,
                "threshold": self.overall_threshold,
                "verdicts": judge_result.get("verdicts", []),
            }
            _metadata: Final = request_data.setdefault("metadata", {})
            existing: Final = _metadata.get("eval_information")
            if isinstance(existing, list):
                existing.append(eval_info)
            elif existing is not None:
                _metadata["eval_information"] = [existing, eval_info]
            else:
                _metadata["eval_information"] = eval_info

            if not passed:
                status = "guardrail_intervened"
                if self.on_failure == "block":
                    raise HTTPException(
                        status_code=422,
                        detail={
                            "error": "LLM judge rejected response: score below threshold",
                            "overall_score": overall_score,
                            "threshold": self.overall_threshold,
                            "verdicts": judge_result.get("verdicts", []),
                        },
                    )

            return inputs

        except HTTPException:
            raise
        except Exception as e:
            verbose_logger.warning("llm_as_a_judge guardrail unexpected error: %s", e)
            return inputs
        finally:
            self.add_standard_logging_guardrail_information_to_request_data(

View on GitHub (pinned to 77b7c6c40c)

Solutions

Lower overall_threshold in litellm_params to a value the judged model can realistically meet
Set on_failure: 'log' to record judge failures in metadata.eval_information without blocking responses
Improve the underlying model or its system prompt so responses satisfy the judge criteria
Rebalance criterion weights so critical criteria dominate and minor ones cannot drag the overall score under the threshold

Example fix

# before - default threshold 80 blocks this model's typical scores
litellm_params:
  judge_model: gpt-4o
  criteria: [{name: grounded, weight: 100}]

# after - realistic threshold, failures logged not blocked
litellm_params:
  judge_model: gpt-4o
  criteria: [{name: grounded, weight: 100}]
  overall_threshold: 70
  on_failure: 'log'

Defensive patterns

Strategy: try-catch

Try / catch

import openai  
  
try:  
    resp = client.chat.completions.create(model=judge_model_deployment, messages=msgs)  
except openai.UnprocessableEntityError as e:  
    body = e.body if isinstance(e.body, dict) else {}  
    if body.get("error", "").startswith("LLM judge rejected"):  
        log_judge_failure(body.get("overall_score"), body.get("verdicts"))  
        return regenerate_with_stricter_prompt(msgs)  
    raise

Prevention

Set overall_threshold from measured scores of your real model, not aspiration - run a benchmark set through the judge first
Use on_failure: 'log' in staging to gather score distributions before enabling 'block' in production
Watch judge latency and cost: every request is judged, so consider judging a sample instead of all traffic

When it happens

Trigger: A completion routed through a deployment with the llm_as_a_judge guardrail where the judged response's weighted score lands under overall_threshold (e.g. 74 vs threshold 80) while on_failure is 'block'. The detail body contains overall_score, threshold, and verdicts showing which criteria failed.

Common situations: Quality gates fronting cheap/small models whose answers fail judge criteria; thresholds set optimistically at 80+; judge criteria phrased so a whole class of valid answers scores low; double latency and cost per request because every response is judged.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/46b36eb41db5fc98. Report an issue: GitHub.