headroomlabs-ai/headroom · error · ImportError

openai package required. Install with: pip install openai

Error message

openai package required. Install with: pip install openai

What it means

Raised by BeforeAfterRunner._init_llm_client when llm_config.provider == 'openai' but the openai SDK is not installed. Like the other provider branches, the import is deferred until the LLM judge is first needed, so the error surfaces mid-run. Note this specific error is about the DIRECT OpenAI path — the proxy path (_init_proxy_client) imports openai unconditionally and would fail with a plain ImportError instead.

Source

Thrown at headroom/evals/runners/before_after.py:114

    def _init_llm_client(self) -> Any:
        """Initialize the appropriate LLM client."""
        if self.llm_config.provider == "anthropic":
            try:
                import anthropic

                return anthropic.Anthropic()
            except ImportError as e:
                raise ImportError(
                    "anthropic package required. Install with: pip install anthropic"
                ) from e
        elif self.llm_config.provider == "openai":
            try:
                import openai

                return openai.OpenAI()
            except ImportError as e:
                raise ImportError(
                    "openai package required. Install with: pip install openai"
                ) from e
        elif self.llm_config.provider == "ollama":
            try:
                import ollama

                return ollama.Client()
            except ImportError as e:
                raise ImportError(
                    "ollama package required. Install with: pip install ollama"
                ) from e
        else:
            raise ValueError(f"Unknown provider: {self.llm_config.provider}")

    def _init_proxy_client(self) -> Any:
        """Initialize an OpenAI client pointing at the Headroom proxy."""
        import openai

View on GitHub (pinned to 322425c43b)

Solutions

1. python -m pip install openai in the interpreter running the eval.
2. Or add the openai extra / eval extra to your project install so CI always has it.
3. If you meant to go through the Headroom proxy instead of direct OpenAI, configure the proxy path so no direct provider SDK is needed.
4. Check: python -c "import openai; print(openai.__version__)".

Example fix

# before
# provider="openai" with no SDK -> ImportError: openai package required

# after
$ python -m pip install openai
# and keep OPENAI_API_KEY exported for the judge

import importlib.util

if importlib.util.find_spec("openai") is None:
    raise SystemExit("openai SDK missing — run: pip install openai")

try:
    runner.run()
except ImportError as e:
    if "openai package required" in str(e):
        sys.exit("Install openai, or set provider='ollama'/'anthropic' if that's your stack")
    raise

When it happens

Trigger: provider='openai' in the before/after runner's LLM config while the active environment lacks the openai package; common when the eval suite was installed with only some provider extras, or the run happens in a fresh tox/nox env.

Common situations: Minimal eval installs; mixing system Python and a project venv; CI matrices that skip the openai extra on some jobs; stale lockfiles after the provider list changed.

Related errors

• anthropic package required. Install with: pip install anthro
• OpenAI API key required. Set OPENAI_API_KEY environment vari
• ollama package required. Install with: pip install ollama
• openai package required. Install with: pip install openai
• Unknown provider: {self.llm_config.provider}

AI-assisted analysis of headroomlabs-ai/headroom@322425c43b (2026-08-15). Data as JSON: /api/errors/6a67890ecaea9431. Report an issue: GitHub.