{"record":{"id":"1afc3fe83200b44c","repo":"headroomlabs-ai/headroom","slug":"unsupported-headroom-bench-proxy-mode-raw-mode-r","errorCode":null,"errorMessage":"unsupported headroom-bench proxy mode: {raw_mode!r}","messagePattern":"unsupported headroom-bench proxy mode: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"headroom/testing/harness.py","lineNumber":1170,"sourceCode":"        if self.proxy_config.disable_kompress:\n            cmd.append(\"--disable-kompress\")\n        if self.proxy_config.lossless:\n            cmd.append(\"--lossless\")\n        if self.proxy_config.bedrock_region:\n            cmd += [\"--bedrock-region\", self.proxy_config.bedrock_region]\n        if self.proxy_config.bedrock_profile:\n            cmd += [\"--bedrock-profile\", self.proxy_config.bedrock_profile]\n        cmd += list(extra_flags)\n        return tuple(cmd)\n\n    def bench_arms(\n        self, *, provider: Literal[\"anthropic\", \"openai\"] = \"anthropic\"\n    ) -> tuple[BenchArm, ...]:\n        headroom_proxy_mode: Literal[\"off\", \"token\", \"cache\"]\n        if self.proxy_config.optimize:\n            raw_mode = cast(str, self.proxy_config.mode)\n            if raw_mode not in {\"token\", \"cache\"}:\n                raise ValueError(f\"unsupported headroom-bench proxy mode: {raw_mode!r}\")\n            headroom_proxy_mode = cast(Literal[\"token\", \"cache\"], raw_mode)\n        else:\n            headroom_proxy_mode = \"off\"\n        return (\n            BenchArm(\n                name=ArmName.A0_DIRECT,\n                provider=provider,\n                proxy_mode=None,\n                proxy_flags=(),\n                label=\"Direct provider API\",\n            ),\n            BenchArm(\n                name=ArmName.A1_PASSTHROUGH,\n                provider=provider,\n                proxy_mode=\"off\",\n                proxy_flags=(),\n                label=\"Headroom proxy passthrough\",\n            ),","sourceCodeStart":1152,"sourceCodeEnd":1188,"githubUrl":"https://github.com/headroomlabs-ai/headroom/blob/322425c43bffde1ed0b64fecf3cf5951565dd82b/headroom/testing/harness.py#L1152-L1188","documentation":"Raised by HarnessScenario.bench_arms() when the proxy is enabled (proxy_config.optimize is true) but proxy_config.mode is neither 'token' nor 'cache'. The bench harness maps the proxy mode onto headroom-bench arms (A0 direct, plus optimization arms) and only token/cache are meaningful benchmark modes, so anything else is rejected.","triggerScenarios":"Building a scenario with configure_proxy(optimize=True) but leaving mode at its default (or setting mode='off', 'auto', 'kompress', etc.) and then calling scenario.bench_arms() or running the suite's benchmark path.","commonSituations":"Copying a proxy config from the main proxy server (which supports more mode strings) into a bench scenario; enabling optimize without deciding which optimization to measure; version changes that renamed proxy modes.","solutions":["Set an explicit supported mode: configure_proxy(optimize=True, mode='token') or mode='cache'.","If you intended no proxy optimization, set optimize=False so bench_arms uses the 'off' arm.","Validate mode membership {'token','cache'} in test setup when optimize is enabled."],"exampleFix":"# before\nheadroom.configure_proxy(optimize=True)  # mode unset/invalid\narms = scenario.bench_arms()\n\n# after\nheadroom.configure_proxy(optimize=True, mode=\"cache\")\narms = scenario.bench_arms()","handlingStrategy":"validation","validationCode":"if proxy_config.optimize:\n    assert proxy_config.mode in {\"token\", \"cache\"}, (\n        f\"bench requires proxy mode 'token' or 'cache', got {proxy_config.mode!r}\")","typeGuard":"def bench_ready(cfg) -> bool:\n    return (not cfg.optimize) or cfg.mode in {\"token\", \"cache\"}","tryCatchPattern":"try:\n    arms = scenario.bench_arms()\nexcept ValueError as e:\n    if \"unsupported headroom-bench proxy mode\" in str(e):\n        scenario.proxy_config.mode = \"token\"\n        arms = scenario.bench_arms()\n    else:\n        raise","preventionTips":["Always set mode explicitly when enabling optimize for benchmarks.","Add a unit test asserting bench_arms() succeeds for every suite scenario.","Keep proxy-mode vocabulary for bench limited to token/cache."],"tags":["benchmark","config","proxy","validation"],"backgroundTag":null,"analyzedSha":"322425c43bffde1ed0b64fecf3cf5951565dd82b","analyzedAt":"2026-08-15T01:03:05.481Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}