{"record":{"id":"8bc6ce0e47586e60","repo":"headroomlabs-ai/headroom","slug":"error-memory-eval-dependencies-not-installed","errorCode":null,"errorMessage":"Error: Memory eval dependencies not installed.","messagePattern":"Error: Memory eval dependencies not installed\\.","errorType":"console","errorClass":"SystemExit","httpStatus":null,"severity":"error","filePath":"headroom/cli/evals.py","lineNumber":454,"sourceCode":"\n    warnings.filterwarnings(\"ignore\", message=\".*Pydantic serializer warnings.*\")\n    warnings.filterwarnings(\"ignore\", category=UserWarning, module=\"pydantic\")\n\n    try:\n        from headroom.evals.memory import (\n            LoCoMoEvaluator,\n            MemoryEvalConfig,\n            create_anthropic_judge,\n            create_litellm_judge,\n            create_openai_judge,\n            simple_judge,\n        )\n        from headroom.memory import MemoryConfig\n    except ImportError as e:\n        click.echo(\"Error: Memory eval dependencies not installed.\")\n        click.echo(\"Run: pip install headroom[memory,evals]\")\n        click.echo(f\"Details: {e}\")\n        raise SystemExit(1) from None\n\n    import asyncio\n\n    # Build configuration\n    parsed_categories = _parse_categories(categories)\n\n    memory_config = MemoryConfig()\n\n    eval_config = MemoryEvalConfig(\n        n_conversations=n_conversations,\n        categories=parsed_categories,\n        skip_adversarial=not include_adversarial,\n        top_k_memories=top_k,\n        llm_judge_enabled=llm_judge,\n        llm_judge_model=judge_model,\n        memory_config=memory_config,\n        f1_threshold=f1_threshold,\n        extract_memories=not no_extract,","sourceCodeStart":436,"sourceCodeEnd":472,"githubUrl":"https://github.com/headroomlabs-ai/headroom/blob/322425c43bffde1ed0b64fecf3cf5951565dd82b/headroom/cli/evals.py#L436-L472","documentation":"The 'headroom evals memory' CLI command imports its evaluator stack (LoCoMoEvaluator, MemoryEvalConfig, judge factories, headroom.memory) inside the command. If any of these optional memory/evals dependencies is missing, the ImportError is caught, a message pointing at 'pip install headroom[memory,evals]' is printed, and the process exits with code 1 via SystemExit. It is a dependency-guard exit, not a data error.","triggerScenarios":"Running `headroom evals memory ...` in an environment without the [memory,evals] extras (e.g. missing lancedb, eval datasets, or judge SDK deps); the failing import name is shown in 'Details: {e}'.","commonSituations":"Bare pip install headroom-ai; CI images trimmed of optional extras; Python version where an eval dependency has no wheel; mixing pip and system site-packages so the extra resolves in a different interpreter.","solutions":["Install both extras: pip install 'headroom-ai[memory,evals]'","Read the 'Details:' line to see exactly which module failed and install it directly if the extra name differs across versions","Confirm you are in the same virtualenv headroom runs from (which headroom; which python)","Upgrade headroom so the extras metadata matches the imports the CLI performs"],"exampleFix":"# before\n$ headroom evals memory --n-conversations 2\n# Error: Memory eval dependencies not installed.\n\n# after\n$ pip install 'headroom-ai[memory,evals]'\n$ headroom evals memory --n-conversations 2","handlingStrategy":"validation","validationCode":"import subprocess, sys\nmissing = subprocess.run(\n    [sys.executable, \"-c\", \"import headroom.evals.memory\"],\n    capture_output=True,\n).returncode != 0\nif missing:\n    raise SystemExit(\"install extras first: pip install 'headroom-ai[memory,evals]'\")","typeGuard":null,"tryCatchPattern":"proc = subprocess.run([\"headroom\", \"evals\", \"memory\", ...], capture_output=True, text=True)\nif proc.returncode != 0 and \"not installed\" in proc.stderr + proc.stdout:\n    print(\"eval extras missing in runner env — install and retry\")","preventionTips":["Pin 'headroom-ai[memory,evals]' in the environment where evals run","Run a dry dependency check in CI before the eval stage","Keep eval environments separate from runtime environments to avoid accidental uninstalling"],"tags":["cli","evals","optional-dependencies","importerror","system-exit"],"backgroundTag":null,"analyzedSha":"322425c43bffde1ed0b64fecf3cf5951565dd82b","analyzedAt":"2026-08-15T01:03:05.481Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}