{"record":{"id":"3218fb77c84e478b","repo":"headroomlabs-ai/headroom","slug":"error-litellm-required-for-answer-model-run-p","errorCode":null,"errorMessage":"Error: litellm required for --answer-model. Run: pip install litellm","messagePattern":"Error: litellm required for --answer-model\\. Run: pip install litellm","errorType":"console","errorClass":"SystemExit","httpStatus":null,"severity":"error","filePath":"headroom/cli/evals.py","lineNumber":520,"sourceCode":"3. For \"when\" questions: give the specific date if mentioned (e.g., \"7 May 2023\", \"2022\")\n4. For \"what\" questions: give the specific thing/action\n5. For \"who\" questions: give the name\n6. If the exact answer is in the memories, use those exact words/dates\n7. If you cannot find the answer, say \"Information not found\"\n\n## Answer (be concise - just the facts):\"\"\"\n\n                response = litellm.completion(\n                    model=answer_model,\n                    messages=[{\"role\": \"user\", \"content\": prompt}],\n                    temperature=0.0,\n                    max_tokens=150,\n                )\n                return response.choices[0].message.content or \"\"\n\n        except ImportError:\n            click.echo(\"Error: litellm required for --answer-model. Run: pip install litellm\")\n            raise SystemExit(1) from None\n\n    # Create LLM judge if enabled\n    llm_judge_fn: Callable[[str, str, str], tuple[float, str]] | None = None\n    if llm_judge:\n        # Use answer model for judge if not explicitly set\n        effective_judge_model = judge_model\n        if answer_model and judge_model == \"gpt-4o\":\n            effective_judge_model = answer_model  # Match the answer model\n\n        if judge_provider == \"simple\":\n            llm_judge_fn = simple_judge\n        elif judge_provider == \"openai\":\n            llm_judge_fn = create_openai_judge(model=effective_judge_model)\n        elif judge_provider == \"anthropic\":\n            llm_judge_fn = create_anthropic_judge(model=effective_judge_model)\n        else:\n            llm_judge_fn = create_litellm_judge(model=effective_judge_model)\n","sourceCodeStart":502,"sourceCodeEnd":538,"githubUrl":"https://github.com/headroomlabs-ai/headroom/blob/322425c43bffde1ed0b64fecf3cf5951565dd82b/headroom/cli/evals.py#L502-L538","documentation":"When the memory-eval command is given --answer-model, it answers questions via litellm.completion. litellm is optional; if importing it fails, the command prints 'Error: litellm required for --answer-model. Run: pip install litellm' and exits 1. Only the --answer-model code path needs it — evals without an answer model run fine.","triggerScenarios":"Running `headroom evals memory --answer-model gpt-4o-mini ...` (or --judge-provider litellm paths) without litellm installed in the active environment.","commonSituations":"Installing [evals] extra but not litellm (it is a separate dependency); running evals on a slim CI image; uninstalling litellm to slim an image and forgetting this path.","solutions":["pip install litellm (or reinstall with an extra that includes it)","Drop --answer-model to use the default answering path that does not require litellm","Set your API key env vars (OPENAI_API_KEY etc.) once litellm is installed so the completion call succeeds","Verify: python -c \"import litellm; print(litellm.__version__)\""],"exampleFix":"# before\n$ headroom evals memory --answer-model gpt-4o-mini\n# Error: litellm required for --answer-model.\n\n# after\n$ pip install litellm\n$ headroom evals memory --answer-model gpt-4o-mini","handlingStrategy":"validation","validationCode":"try:\n    import litellm  # noqa: F401\n    HAS_LITELLM = True\nexcept ImportError:\n    HAS_LITELLM = False\n\nif answer_model and not HAS_LITELLM:\n    raise SystemExit(\"--answer-model requires litellm: pip install litellm\")","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Add litellm to your eval requirements whenever --answer-model is part of the workflow","Wrap eval invocations in scripts and check for exit 1 with the litellm message to fail fast","Set model API keys in the same script that runs evals"],"tags":["cli","evals","litellm","optional-dependencies"],"backgroundTag":null,"analyzedSha":"322425c43bffde1ed0b64fecf3cf5951565dd82b","analyzedAt":"2026-08-15T01:03:05.481Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}