{"record":{"id":"fe5e2f922dd0d09e","repo":"headroomlabs-ai/headroom","slug":"litellm-package-required-for-llm-judge-install-wi","errorCode":null,"errorMessage":"LiteLLM package required for LLM judge. Install with: pip install litellm","messagePattern":"LiteLLM package required for LLM judge\\. Install with: pip install litellm","errorType":"exception","errorClass":"ImportError","httpStatus":null,"severity":"error","filePath":"headroom/evals/memory/judge.py","lineNumber":170,"sourceCode":") -> Callable[[str, str, str], tuple[float, str]]:\n    \"\"\"Create an LLM judge using LiteLLM for any supported provider.\n\n    Args:\n        model: Model identifier (e.g., \"gpt-4o\", \"claude-sonnet-4-20250514\", \"ollama/llama3\").\n\n    Returns:\n        A judge function that takes (question, ground_truth, prediction)\n        and returns (score, reasoning).\n\n    Example:\n        # Use Ollama for local evaluation\n        judge_fn = create_litellm_judge(model=\"ollama/llama3\")\n        score, reasoning = judge_fn(question, ground_truth, prediction)\n    \"\"\"\n    try:\n        import litellm\n    except ImportError as e:\n        raise ImportError(\n            \"LiteLLM package required for LLM judge. Install with: pip install litellm\"\n        ) from e\n\n    def judge(question: str, ground_truth: str, prediction: str) -> tuple[float, str]:\n        prompt = JUDGE_PROMPT.format(\n            question=question,\n            ground_truth=ground_truth,\n            prediction=prediction,\n        )\n\n        response = litellm.completion(\n            model=model,\n            messages=[{\"role\": \"user\", \"content\": prompt}],\n            temperature=0.0,\n            max_tokens=200,\n        )\n\n        text = response.choices[0].message.content or \"\"","sourceCodeStart":152,"sourceCodeEnd":188,"githubUrl":"https://github.com/headroomlabs-ai/headroom/blob/322425c43bffde1ed0b64fecf3cf5951565dd82b/headroom/evals/memory/judge.py#L152-L188","documentation":"Error \"LiteLLM package required for LLM judge. Install with: pip install litellm\" thrown in headroomlabs-ai/headroom.","triggerScenarios":"Raised when the memory eval LLM judge routes through LiteLLM but the `litellm` package is not installed.","commonSituations":"See trigger scenarios.","solutions":["Install LiteLLM: pip install litellm","Or install the evals extra: pip install headroom-ai[evals]"],"exampleFix":null,"handlingStrategy":null,"validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"322425c43bffde1ed0b64fecf3cf5951565dd82b","analyzedAt":"2026-08-15T01:03:05.481Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}