headroomlabs-ai/headroom · error · SystemExit

Error: Memory eval V2 dependencies not installed.

Error message

Error: Memory eval V2 dependencies not installed.

What it means

The V2 memory eval command (headroom evals memory-v2 or equivalent) imports LoCoMoEvaluatorV2 / MemoryEvalConfigV2 from headroom.evals.memory inside the command. If those imports fail (optional memory/evals deps missing on this install), the ImportError is caught, a message pointing at 'pip install headroom[memory,evals]' plus the underlying 'Details:' is printed, and the process exits 1. Same dependency guard as the V1 path but for the V2 evaluator stack.

Source

Thrown at headroom/cli/evals.py:626

    debug: bool,
) -> None:
    """Run LoCoMo V2 memory evaluation (LLM-controlled tools)."""
    # Suppress noisy pydantic warnings from litellm
    import warnings

    warnings.filterwarnings("ignore", message=".*Pydantic serializer warnings.*")
    warnings.filterwarnings("ignore", category=UserWarning, module="pydantic")

    try:
        from headroom.evals.memory import (
            LoCoMoEvaluatorV2,
            MemoryEvalConfigV2,
        )
    except ImportError as e:
        click.echo("Error: Memory eval V2 dependencies not installed.")
        click.echo("Run: pip install headroom[memory,evals]")
        click.echo(f"Details: {e}")
        raise SystemExit(1) from None

    import asyncio

    # Build configuration
    parsed_categories = _parse_categories(categories)

    eval_config = MemoryEvalConfigV2(
        n_conversations=n_conversations,
        categories=parsed_categories,
        skip_adversarial=not include_adversarial,
        llm_judge_enabled=llm_judge,
        llm_judge_model=judge_model,
        f1_threshold=f1_threshold,
        parallel_workers=parallel,
        debug=debug,
        save_model=save_model,
        answer_model=answer_model,
        max_search_results=max_results,

View on GitHub (pinned to 322425c43b)

Solutions

1. pip install 'headroom-ai[memory,evals]'
2. Check the 'Details:' line for the exact missing module and pip install it by name
3. Confirm the active interpreter/venv is the one headroom was installed into
4. If V2 modules are absent in your headroom version, upgrade headroom-ai

Example fix

# before
$ headroom evals memory-v2
# Error: Memory eval V2 dependencies not installed.

# after
$ pip install 'headroom-ai[memory,evals]' --upgrade
$ headroom evals memory-v2

try:
    from headroom.evals.memory import LoCoMoEvaluatorV2  # noqa: F401
    V2_OK = True
except ImportError:
    V2_OK = False

if not V2_OK:
    raise SystemExit("pip install 'headroom-ai[memory,evals]' before running V2 evals")

When it happens

Trigger: Running the V2 memory eval CLI subcommand in an environment where headroom.evals.memory cannot import (missing extras, or a headroom version where the V2 modules live behind an extra).

Common situations: Bare headroom-ai install; CI images without extras; upgrading headroom and forgetting to reinstall extras; Python version where an eval dep lacks a wheel.

Related errors

• Error: Memory eval dependencies not installed.
• Error: litellm required for --answer-model. Run: pip install
• Error: MCP SDK not installed.
• Error: MCP dependencies not installed: {e}
• Error: Proxy dependencies not installed. Run: pip install he

AI-assisted analysis of headroomlabs-ai/headroom@322425c43b (2026-08-15). Data as JSON: /api/errors/209079f93d21b491. Report an issue: GitHub.