headroomlabs-ai/headroom · error · ImportError

anthropic package required. Install with: pip install anthro

Error message

anthropic package required. Install with: pip install anthropic

What it means

Raised by BeforeAfterRunner._init_llm_client when llm_config.provider == 'anthropic' but the anthropic Python package is not importable in the current environment. The eval runner lazily imports the provider SDK only at judge-initialization time, so the failure appears at first ground-truth evaluation, not at runner construction. It is an optional-dependency error with an explicit pip remedy in the message.

Source

Thrown at headroom/evals/runners/before_after.py:105

        self._proxy_client: Any = None
        if self.llm_config.headroom_proxy_url:
            self._proxy_client = self._init_proxy_client()

        # ContentRouter is still used as fallback when no proxy is configured
        self._router = ContentRouter(config=self.router_config)

        # Lazy-initialized LLM judge for ground-truth evaluation
        self._judge_fn: Any = None

    def _init_llm_client(self) -> Any:
        """Initialize the appropriate LLM client."""
        if self.llm_config.provider == "anthropic":
            try:
                import anthropic

                return anthropic.Anthropic()
            except ImportError as e:
                raise ImportError(
                    "anthropic package required. Install with: pip install anthropic"
                ) from e
        elif self.llm_config.provider == "openai":
            try:
                import openai

                return openai.OpenAI()
            except ImportError as e:
                raise ImportError(
                    "openai package required. Install with: pip install openai"
                ) from e
        elif self.llm_config.provider == "ollama":
            try:
                import ollama

                return ollama.Client()
            except ImportError as e:
                raise ImportError(

View on GitHub (pinned to 322425c43b)

Solutions

1. pip install anthropic in the SAME interpreter that runs the eval (python -m pip install anthropic to be sure).
2. Or install the project's eval extras (e.g. pip install -e '.[evals]') which should pin provider deps.
3. If you did not intend Anthropic, switch llm_config.provider to a provider whose SDK you have (openai/ollama).
4. Verify with: python -c "import anthropic; print(anthropic.__version__)".

Example fix

# before
# LLMJudgeConfig(provider="anthropic") in a bare env -> ImportError at first judge call

# after
$ python -m pip install anthropic
# or pin via extras in pyproject: [project.optional-dependencies] evals = ["anthropic>=0.30"]

import importlib.util, sys

missing = [p for p in ("anthropic",) if importlib.util.find_spec(p) is None]
if missing:
    raise SystemExit(f"install provider SDKs first: pip install {' '.join(missing)}")

try:
    runner = BeforeAfterRunner(cfg)
    runner.run()
except ImportError as e:
    if "anthropic" in str(e):
        sys.exit("pip install anthropic (or switch provider) then rerun")
    raise

When it happens

Trigger: Configuring LLMJudgeConfig(provider='anthropic') (or a suite YAML selecting anthropic) and starting a before/after eval run in an environment where 'pip install anthropic' was never executed or the package lives in a different virtualenv than the one running the eval.

Common situations: Using the headroom[evals]/eval extra that does not pin every provider SDK; activating the wrong venv/conda env; a Docker image built without provider extras; CI cache serving a stale env.

Related errors

• openai package required. Install with: pip install openai
• ollama package required. Install with: pip install ollama
• anthropic package required. Install with: pip install anthro
• OpenAI API key required. Set OPENAI_API_KEY environment vari
• Unknown provider: {self.llm_config.provider}

AI-assisted analysis of headroomlabs-ai/headroom@322425c43b (2026-08-15). Data as JSON: /api/errors/661bef5b72112911. Report an issue: GitHub.