BerriAI/litellm · error · OpenAIError

error.get("message")

Error message

error.get("message")

What it means

This is the error-forwarding site in GroqChatCompletionStreamingHandler.chunk_parser: when a streamed SSE chunk from Groq contains a top-level "error" object, LiteLLM re-raises it as an OpenAIError carrying the upstream status code, message, and raw body. So the message you see is Groq's own error text (error.get("message")), surfaced mid-stream by LiteLLM — not a bug in the line itself.

Source

Thrown at litellm/llms/groq/chat/transformation.py:352

            return ()

    def _map_groq_service_tier(self, original_service_tier: str | None) -> Literal["auto", "default", "flex"]:
        """
        Ensure groq service tier is OpenAI compatible.
        """
        if original_service_tier is None:
            return "auto"
        if original_service_tier not in ["auto", "default", "flex"]:
            return "auto"

        return cast(Literal["auto", "default", "flex"], original_service_tier)


class GroqChatCompletionStreamingHandler(OpenAIChatCompletionStreamingHandler):
    def chunk_parser(self, chunk: dict) -> ModelResponseStream:
        error: Final = chunk.get("error")
        if error:
            raise OpenAIError(status_code=error.get("code"), message=error.get("message"), body=error)

        # Map Groq's 'reasoning' field to LiteLLM's 'reasoning_content' field
        # Groq returns delta.reasoning, but LiteLLM expects delta.reasoning_content
        choices: Final = chunk.get("choices", [])
        for choice in choices:
            delta = choice.get("delta", {})
            if "reasoning" in delta:
                delta["reasoning_content"] = delta.pop("reasoning")

        return super().chunk_parser(chunk)

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Read the surfaced message/code: 401/403 → fix GROQ_API_KEY; 404 → fix the model name against Groq's current model list; 429 → slow down, add rate limiting, or upgrade tier; 5xx/overloaded → retry with backoff.
2. For transient errors, retry the streaming call with exponential backoff (litellm supports num_retries, or wrap your loop).
3. Guard upstream: validate model ids when Groq deprecates models, and pin versions you have tested.
4. If errors persist with a valid key and model, check Groq status page and Groq console for outage/quota issues.

Example fix

# before
response = litellm.completion(model='groq/llama3-70b-8192', messages=msgs, stream=True)
for chunk in response: ...  # raises OpenAIError mid-stream on Groq-side error

# after
try:
    response = litellm.completion(
        model='groq/llama-3.3-70b-versatile', messages=msgs, stream=True, num_retries=3
    )
    for chunk in response:
        ...
except litellm.exceptions.OpenAIError as e:
    # inspect e.status_code: 401 fix key, 404 fix model, 429 back off, 5xx retry later
    raise

try:
    stream = litellm.completion(model="groq/...", messages=msgs, stream=True, num_retries=3)
    for chunk in stream:
        ...
except litellm.exceptions.OpenAIError as e:
    code = getattr(e, "status_code", None)
    if code == 404:
        raise RuntimeError("Bad groq model id") from e
    if code == 429 or code >= 500:
        backoff_and_retry()  # transient
    raise

When it happens

Trigger: Streaming a Groq completion (stream=True) that fails after the stream opens: model name typo/undeployed model, rate limits, overloaded Groq capacity, or an invalid/revoked GROQ_API_KEY — Groq emits {"error": {"code": ..., "message": ...}} as a chunk and this handler raises.

Common situations: Using model='groq/llama-3.x-...' with a model id that was deprecated or renamed; hitting Groq free-tier rate limits during bursts; expired API key that only fails at stream time; transient 5xx/overloads on Groq's side.

Related errors

• {e.response.text}
• Braintrust API error: {e.response.text}
• Failed to connect to Braintrust API: {str(e)}
• Project tpm_limit ({data.tpm_limit}) exceeds team's tpm_limi
• Project rpm_limit ({data.rpm_limit}) exceeds team's rpm_limi

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/f1a3b6a185c84669. Report an issue: GitHub.