{"record":{"id":"f5f5fb356021e4f2","repo":"headroomlabs-ai/headroom","slug":"scenarioorchestrator-requires-at-least-one-scenari","errorCode":null,"errorMessage":"ScenarioOrchestrator requires at least one scenario","messagePattern":"ScenarioOrchestrator requires at least one scenario","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"headroom/testing/harness.py","lineNumber":1524,"sourceCode":"            time.sleep(0.25)\n        raise TimeoutError(f\"headroom proxy was not ready at {url}: {last_error}\")\n\n\nclass _NoopClient:\n    \"\"\"Minimal original-client object for SDK simulations without upstream I/O.\"\"\"\n\n\nclass ScenarioOrchestrator:\n    \"\"\"Runs built scenarios against local no-key tasks and evaluates guarantees.\"\"\"\n\n    def __init__(\n        self,\n        scenarios: Sequence[HarnessScenario],\n        *,\n        guarantees: Sequence[Guarantee] = DEFAULT_GUARANTEES,\n    ) -> None:\n        if not scenarios:\n            raise ValueError(\"ScenarioOrchestrator requires at least one scenario\")\n        self._scenarios = tuple(scenarios)\n        self._guarantees = tuple(guarantees)\n\n    def run(self, tasks: Sequence[ScenarioTask]) -> ScenarioRunReport:\n        if not tasks:\n            raise ValueError(\"ScenarioOrchestrator.run requires at least one task\")\n\n        cases: list[ScenarioCaseResult] = []\n        for scenario in self._scenarios:\n            for task in tasks:\n                result = scenario.simulate(\n                    task.messages,\n                    model=task.model,\n                    provider=task.provider,\n                    output_buffer_tokens=task.output_buffer_tokens,\n                )\n                guarantees = tuple(\n                    guarantee(scenario, task, result) for guarantee in self._guarantees","sourceCodeStart":1506,"sourceCodeEnd":1542,"githubUrl":"https://github.com/headroomlabs-ai/headroom/blob/322425c43bffde1ed0b64fecf3cf5951565dd82b/headroom/testing/harness.py#L1506-L1542","documentation":"Raised by ScenarioOrchestrator.__init__ when constructed with an empty scenarios sequence. The orchestrator's job is to cross scenarios with tasks and evaluate guarantees, so an empty scenario list is treated as a caller bug and rejected immediately rather than producing an empty report.","triggerScenarios":"ScenarioOrchestrator([]) or ScenarioOrchestrator(suite.scenarios) where the suite had no scenarios; passing a filtered/generator-derived sequence that yielded nothing; constructing the orchestrator before adding scenarios to the suite and never re-creating it.","commonSituations":"Parameterized test configs selecting zero scenarios via env filter; refactors converting a list comprehension to a generator consumed elsewhere; suite built but scenarios added after orchestrator creation.","solutions":["Pass at least one scenario: orchestrator = ScenarioOrchestrator([scenario]) or ScenarioOrchestrator(suite.scenarios) after suite.add(...).","If scenarios are dynamic, assert non-empty before constructing: `assert scenarios, 'no scenarios selected'`.","Check the filter/env var that produced the empty sequence."],"exampleFix":"# before\norch = ScenarioOrchestrator([])  # ValueError\n\n# after\norch = ScenarioOrchestrator([scenario_a, scenario_b])","handlingStrategy":"validation","validationCode":"assert scenarios, \"ScenarioOrchestrator needs >=1 scenario\"\norch = ScenarioOrchestrator(scenarios)","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Check suite.scenarios non-empty before constructing the orchestrator.","Fail test setup early when scenario filters select nothing.","Construct the orchestrator after the suite is fully populated."],"tags":["testing","validation","empty-input","orchestrator"],"backgroundTag":null,"analyzedSha":"322425c43bffde1ed0b64fecf3cf5951565dd82b","analyzedAt":"2026-08-15T01:03:05.481Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}