BerriAI/litellm · error · ValueError

{json.dumps(event_obj)}

Error message

{json.dumps(event_obj)}

What it means

While normalizing SAP Generative AI Hub orchestration stream events into OpenAI-style chunks, LiteLLM treats any event object carrying a 'code' or 'error' key as an in-stream error per the orchestration spec, and raises ValueError containing the full event as JSON. This is how mid-stream failures (moderation blocks, downstream model errors) are surfaced instead of being silently skipped.

Source

Thrown at litellm/llms/sap/chat/handler.py:88

                    for c in (orc.get("choices") or [])
                ],
            }
        )

    @staticmethod
    def to_openai_chunk(event_obj: dict) -> OpenAIChatCompletionChunk | None:
        """
        Accepts:
          - {"final_result": <openai-style CHUNK>}   (IMPORTANT: this is just another chunk, NOT terminal)
          - {"orchestration_result": {...}}          (map to chunk)
          - already-openai-shaped chunks
          - other events (ignored)
        Raises:
          - ValueError for in-stream error objects
        """
        # In-stream error per spec (surface as exception)
        if "code" in event_obj or "error" in event_obj:
            raise ValueError(json.dumps(event_obj))

        # FINAL RESULT IS *NOT* TERMINAL: treat it as the next chunk
        if "final_result" in event_obj:
            fr: Final = event_obj["final_result"] or {}
            # ensure it looks like an OpenAI chunk
            if "object" not in fr:
                fr["object"] = "chat.completion.chunk"
            return OpenAIChatCompletionChunk.model_validate(fr)

        # Orchestration incremental delta
        if "orchestration_result" in event_obj:
            return _StreamParser._from_orchestration_result(event_obj)

        # Already an OpenAI-like chunk
        if "choices" in event_obj and "object" in event_obj:
            return OpenAIChatCompletionChunk.model_validate(event_obj)

        # Unknown / heartbeat / metrics

View on GitHub (pinned to 77b7c6c40c)

Solutions

1. Parse the JSON in the exception message: it contains the SAP error code and details that identify the failing module.
2. If the code indicates a filtering/masking rejection, adjust the input or the module configuration in the request.
3. If it names a deployment/model issue, verify model names and that the deployment exists in your AI Core resource group.
4. Reproduce with stream=False to see whether the same orchestration error appears as a normal HTTP error response.

import json

try:
    for chunk in stream:
        handle(chunk)
except ValueError as e:
    try:
        sap_err = json.loads(str(e))
    except json.JSONDecodeError:
        raise
    log.error('SAP in-stream error code=%s', sap_err.get('code'))

When it happens

Trigger: Streaming a sap/ chat completion where the orchestration pipeline emits an error event after the stream opened - e.g. the filtering/masking module rejected content, a grounding datasource failed, or the underlying deployment returned an error mid-generation. Also triggered by any non-chunk event that happens to contain 'code' (e.g. metrics events with a code field).

Common situations: Azure OpenAI grounding or templating module misconfigured so the orchestration fails after the stream starts; content filters tripping on user input; deployments where the model id in the module config is wrong and the error only surfaces once generation begins.

Related errors

• {err.args[0]}
• A2A send_message_streaming failed: no response received afte
• api_base is required for Pydantic AI agents
• Stream completed response is invalid
• Chat provider: Empty parsed_chunk

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/2342f30fffd9f4ec. Report an issue: GitHub.