{"record":{"id":"5690d796a4394409","repo":"headroomlabs-ai/headroom","slug":"self-name-backend-does-not-support-openai-stream","errorCode":null,"errorMessage":"{self.name} backend does not support OpenAI streaming","messagePattern":"(.+?) backend does not support OpenAI streaming","errorType":"exception","errorClass":"NotImplementedError","httpStatus":null,"severity":"error","filePath":"headroom/backends/base.py","lineNumber":163,"sourceCode":"        body: dict[str, Any],\n        headers: dict[str, str],\n    ) -> AsyncIterator[str]:\n        \"\"\"Stream an OpenAI-format chat completion.\n\n        Yields SSE-formatted strings: 'data: {...}\\\\n\\\\n' for each chunk,\n        ending with 'data: [DONE]\\\\n\\\\n'.\n\n        Args:\n            body: Request body in OpenAI chat completion format (stream: true).\n            headers: Request headers.\n\n        Yields:\n            SSE-formatted strings ready to send to client.\n\n        Raises:\n            NotImplementedError: If backend doesn't support OpenAI streaming.\n        \"\"\"\n        raise NotImplementedError(f\"{self.name} backend does not support OpenAI streaming\")\n        # Make this an async generator (yield never reached but needed for type)\n        yield \"\"  # type: ignore[misc]  # pragma: no cover\n\n    async def close(self) -> None:  # noqa: B027\n        \"\"\"Clean up resources (e.g., close HTTP clients).\"\"\"\n        pass\n","sourceCodeStart":145,"sourceCodeEnd":170,"githubUrl":"https://github.com/headroomlabs-ai/headroom/blob/322425c43bffde1ed0b64fecf3cf5951565dd82b/headroom/backends/base.py#L145-L170","documentation":"The default Backend.stream_openai_message raises NotImplementedError for backends that do not implement OpenAI-format SSE streaming. The unreachable 'yield \"\"' after the raise exists only so the method is typed as an AsyncIterator[str]; it is never executed. Streaming requests (stream: true) routed to such a backend fail here.","triggerScenarios":"Sending a chat-completion request with \"stream\": true through a Backend subclass that overrides handle_openai but not stream_openai_message, or a backend with no OpenAI support at all.","commonSituations":"A backend implements non-streaming OpenAI format but streaming was never added; a client SDK (many default to streaming) hits a backend that only supports non-streaming calls.","solutions":["Retry the request with \"stream\": false if the backend supports non-streaming OpenAI format (handle_openai).","Use a backend that implements stream_openai_message.","If you own the backend, implement stream_openai_message: yield 'data: {...}\\n\\n' chunks and a final 'data: [DONE]\\n\\n'.","At the proxy layer, force stream=false for backends without streaming support instead of letting the stub raise."],"exampleFix":"# before\nbody = {\"model\": \"m\", \"messages\": [...], \"stream\": True}\nasync for chunk in backend.stream_openai_message(body, headers): ...\n\n# after\nclass MyBackend(Backend):\n    async def stream_openai_message(self, body, headers):\n        async for delta in self._native_stream(body):\n            yield f\"data: {json.dumps(to_openai_chunk(delta))}\\n\\n\"\n        yield \"data: [DONE]\\n\\n\"","handlingStrategy":"type-guard","validationCode":"def supports_openai_streaming(backend: Backend) -> bool:\n    return type(backend).stream_openai_message is not Backend.stream_openai_message\n\nif body.get(\"stream\") and not supports_openai_streaming(backend):\n    body = {**body, \"stream\": False}  # downgrade instead of crashing","typeGuard":"def is_stream_capable(b: object) -> bool:\n    m = getattr(type(b), \"stream_openai_message\", None)\n    return m is not None and getattr(Backend, \"stream_openai_message\", None) is not None and m is not Backend.stream_openai_message","tryCatchPattern":"try:\n    async for chunk in backend.stream_openai_message(body, headers):\n        send(chunk)\nexcept NotImplementedError:\n    logger.warning(\"%s cannot stream; retrying non-streaming\", backend.name)\n    resp = await backend.handle_openai({**body, \"stream\": False}, headers)\n    send(resp.body)","preventionTips":["Force stream=false for backends without a streaming override before the request reaches them.","Note many client SDKs default to streaming — configure them explicitly.","Test streaming and non-streaming paths separately per backend."],"tags":["python","backend","streaming","not-implemented","openai-compat","sse"],"backgroundTag":null,"analyzedSha":"322425c43bffde1ed0b64fecf3cf5951565dd82b","analyzedAt":"2026-08-15T01:03:05.481Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}