headroomlabs-ai/headroom · error · ImportError

Anthropic package required for LLM judge. Install with: pip

Error message

Anthropic package required for LLM judge. Install with: pip install anthropic

What it means

Error "Anthropic package required for LLM judge. Install with: pip install anthropic" thrown in headroomlabs-ai/headroom.

Source

Thrown at headroom/evals/memory/judge.py:123

        model: Anthropic model to use (default: claude-sonnet-4-20250514).
        api_key: Anthropic API key (uses ANTHROPIC_API_KEY env var if not provided).

    Returns:
        A judge function that takes (question, ground_truth, prediction)
        and returns (score, reasoning).

    Example:
        judge_fn = create_anthropic_judge()
        score, reasoning = judge_fn(
            "When did Bob start his new job?",
            "March 2024",
            "Bob began working at his new position in early March of 2024"
        )
    """
    try:
        import anthropic
    except ImportError as e:
        raise ImportError(
            "Anthropic package required for LLM judge. Install with: pip install anthropic"
        ) from e

    client = anthropic.Anthropic(api_key=api_key) if api_key else anthropic.Anthropic()

    def judge(question: str, ground_truth: str, prediction: str) -> tuple[float, str]:
        prompt = JUDGE_PROMPT.format(
            question=question,
            ground_truth=ground_truth,
            prediction=prediction,
        )

        response = client.messages.create(
            model=model,
            max_tokens=200,
            messages=[{"role": "user", "content": prompt}],
        )

View on GitHub (pinned to 322425c43b)

Solutions

1. Install the Anthropic package: pip install anthropic
2. Set ANTHROPIC_API_KEY in the environment before running the judge
3. Or install the evals extra: pip install headroom-ai[evals]

When it happens

Trigger: Raised when the memory eval LLM judge is configured with the Anthropic provider but the `anthropic` package is not installed.

Common situations: See trigger scenarios.

AI-assisted analysis of headroomlabs-ai/headroom@322425c43b (2026-08-15). Data as JSON: /api/errors/4efedddf425102c9. Report an issue: GitHub.