headroomlabs-ai/headroom · error · NotImplementedError

{self.name} backend does not support OpenAI streaming

Error message

{self.name} backend does not support OpenAI streaming

What it means

The default Backend.stream_openai_message raises NotImplementedError for backends that do not implement OpenAI-format SSE streaming. The unreachable 'yield ""' after the raise exists only so the method is typed as an AsyncIterator[str]; it is never executed. Streaming requests (stream: true) routed to such a backend fail here.

Source

Thrown at headroom/backends/base.py:163

        body: dict[str, Any],
        headers: dict[str, str],
    ) -> AsyncIterator[str]:
        """Stream an OpenAI-format chat completion.

        Yields SSE-formatted strings: 'data: {...}\\n\\n' for each chunk,
        ending with 'data: [DONE]\\n\\n'.

        Args:
            body: Request body in OpenAI chat completion format (stream: true).
            headers: Request headers.

        Yields:
            SSE-formatted strings ready to send to client.

        Raises:
            NotImplementedError: If backend doesn't support OpenAI streaming.
        """
        raise NotImplementedError(f"{self.name} backend does not support OpenAI streaming")
        # Make this an async generator (yield never reached but needed for type)
        yield ""  # type: ignore[misc]  # pragma: no cover

    async def close(self) -> None:  # noqa: B027
        """Clean up resources (e.g., close HTTP clients)."""
        pass

View on GitHub (pinned to 322425c43b)

Solutions

1. Retry the request with "stream": false if the backend supports non-streaming OpenAI format (handle_openai).
2. Use a backend that implements stream_openai_message.
3. If you own the backend, implement stream_openai_message: yield 'data: {...}\n\n' chunks and a final 'data: [DONE]\n\n'.
4. At the proxy layer, force stream=false for backends without streaming support instead of letting the stub raise.

Example fix

# before
body = {"model": "m", "messages": [...], "stream": True}
async for chunk in backend.stream_openai_message(body, headers): ...

# after
class MyBackend(Backend):
    async def stream_openai_message(self, body, headers):
        async for delta in self._native_stream(body):
            yield f"data: {json.dumps(to_openai_chunk(delta))}\n\n"
        yield "data: [DONE]\n\n"

def supports_openai_streaming(backend: Backend) -> bool:
    return type(backend).stream_openai_message is not Backend.stream_openai_message

if body.get("stream") and not supports_openai_streaming(backend):
    body = {**body, "stream": False}  # downgrade instead of crashing

def is_stream_capable(b: object) -> bool:
    m = getattr(type(b), "stream_openai_message", None)
    return m is not None and getattr(Backend, "stream_openai_message", None) is not None and m is not Backend.stream_openai_message

try:
    async for chunk in backend.stream_openai_message(body, headers):
        send(chunk)
except NotImplementedError:
    logger.warning("%s cannot stream; retrying non-streaming", backend.name)
    resp = await backend.handle_openai({**body, "stream": False}, headers)
    send(resp.body)

When it happens

Trigger: Sending a chat-completion request with "stream": true through a Backend subclass that overrides handle_openai but not stream_openai_message, or a backend with no OpenAI support at all.

Common situations: A backend implements non-streaming OpenAI format but streaming was never added; a client SDK (many default to streaming) hits a backend that only supports non-streaming calls.

Related errors

• {self.name} backend does not support OpenAI format
• bedrock_upstream_exception
• any-llm-sdk is required for AnyLLMBackend. Install with: pip
• litellm is required for LiteLLMBackend. Install with: pip in
• bedrock_eventstream_missing_event_type

AI-assisted analysis of headroomlabs-ai/headroom@322425c43b (2026-08-15). Data as JSON: /api/errors/5690d796a4394409. Report an issue: GitHub.