{"record":{"id":"3b6877f98ca9459d","repo":"headroomlabs-ai/headroom","slug":"scenarioorchestrator-run-requires-at-least-one-tas","errorCode":null,"errorMessage":"ScenarioOrchestrator.run requires at least one task","messagePattern":"ScenarioOrchestrator\\.run requires at least one task","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"headroom/testing/harness.py","lineNumber":1530,"sourceCode":"\n\nclass ScenarioOrchestrator:\n    \"\"\"Runs built scenarios against local no-key tasks and evaluates guarantees.\"\"\"\n\n    def __init__(\n        self,\n        scenarios: Sequence[HarnessScenario],\n        *,\n        guarantees: Sequence[Guarantee] = DEFAULT_GUARANTEES,\n    ) -> None:\n        if not scenarios:\n            raise ValueError(\"ScenarioOrchestrator requires at least one scenario\")\n        self._scenarios = tuple(scenarios)\n        self._guarantees = tuple(guarantees)\n\n    def run(self, tasks: Sequence[ScenarioTask]) -> ScenarioRunReport:\n        if not tasks:\n            raise ValueError(\"ScenarioOrchestrator.run requires at least one task\")\n\n        cases: list[ScenarioCaseResult] = []\n        for scenario in self._scenarios:\n            for task in tasks:\n                result = scenario.simulate(\n                    task.messages,\n                    model=task.model,\n                    provider=task.provider,\n                    output_buffer_tokens=task.output_buffer_tokens,\n                )\n                guarantees = tuple(\n                    guarantee(scenario, task, result) for guarantee in self._guarantees\n                )\n                cases.append(\n                    ScenarioCaseResult(\n                        scenario=scenario.name,\n                        task_id=task.task_id,\n                        result=result,","sourceCodeStart":1512,"sourceCodeEnd":1548,"githubUrl":"https://github.com/headroomlabs-ai/headroom/blob/322425c43bffde1ed0b64fecf3cf5951565dd82b/headroom/testing/harness.py#L1512-L1548","documentation":"Raised by ScenarioOrchestrator.run() when the tasks argument is an empty sequence. run() cross-products scenarios x tasks to produce ScenarioCaseResults; with no tasks there is nothing to evaluate, so it fails fast with ValueError.","triggerScenarios":"Calling orchestrator.run([]) directly; passing a task list loaded from a directory that was empty (bad glob, wrong fixtures path); tasks filtered out by provider/model predicate before run().","commonSituations":"Test fixtures directory moved or renamed; CI checkout missing data files; a provider filter (e.g. only openai tasks) removing every task in an anthropic-only environment.","solutions":["Supply at least one ScenarioTask: orchestrator.run([ScenarioTask(messages=[...], model=..., provider=...)]).","Verify the fixture path/glob used to build tasks and that files parse (a loader may swallow errors and return []).","Guard in test setup: skip the test when the task list is empty instead of calling run()."],"exampleFix":"# before\nreport = orch.run(tasks=load_tasks(\"./fixtures\"))  # empty dir -> ValueError\n\n# after\ntasks = load_tasks(\"./fixtures\")\nassert tasks, \"no scenario tasks loaded\"\nreport = orch.run(tasks)","handlingStrategy":"validation","validationCode":"if not tasks:\n    pytest.skip(\"no scenario tasks available\")\nreport = orch.run(tasks)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Validate task loaders return non-empty lists and log counts.","Verify fixture paths in CI checkouts.","Skip (don't fail) parameterized runs whose task set is legitimately empty."],"tags":["testing","validation","empty-input","fixtures"],"backgroundTag":null,"analyzedSha":"322425c43bffde1ed0b64fecf3cf5951565dd82b","analyzedAt":"2026-08-15T01:03:05.481Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}