wandb/openui · error · HTTPException

LiteLLM API key is not set.

Error message

LiteLLM API key is not set.

What it means

This error is raised by OpenUI's chat_completions endpoint when a request targets a model prefixed with 'litellm/' but the LiteLLM client was never initialized because the required LiteLLM API key environment variable was absent at startup. The handler detects `litellm is None` and returns HTTP 500. Like the Groq case, it is a local configuration problem rather than a remote API failure.

Source

Thrown at backend/openui/server.py:166

        # Groq Models
        elif data.get("model").startswith("groq/"):
            data["model"] = data["model"].replace("groq/", "")
            if groq is None:
                raise HTTPException(status=500, detail="Groq API key is not set.")
            response: AsyncStream[
                ChatCompletionChunk
            ] = await groq.chat.completions.create(
                **data,
            )
            return StreamingResponse(
                openai_stream_generator(response, input_tokens, user_id, 1),
                media_type="text/event-stream",
            )
        # Litellm Models
        elif data.get("model").startswith("litellm/"):
            data["model"] = data["model"].replace("litellm/", "")
            if litellm is None:
                raise HTTPException(status=500, detail="LiteLLM API key is not set.")
            response: AsyncStream[
                ChatCompletionChunk
            ] = await litellm.chat.completions.create(
                **data,
            )
            return StreamingResponse(
                openai_stream_generator(response, input_tokens, user_id, 1),
                media_type="text/event-stream",
            )
        # Ollama Time
        elif data.get("model").startswith("ollama/"):
            data["model"] = data["model"].replace("ollama/", "")
            data.pop("max_tokens")
            data["messages"] = openai_to_ollama(data)
            ollama_vision_models = ["llava", "moondream"]
            if any([data["model"].startswith(m) for m in ollama_vision_models]):
                # The Ollama OpenAPI compatibility layer doesn't support images
                # see: https://github.com/ollama/ollama/issues/3690

View on GitHub (pinned to 42d7ab4ab6)

Solutions

  1. Set the LITELLM_API_KEY environment variable (and any LiteLLM proxy URL if used) before launching the server, then restart.
  2. Confirm the variable name matches exactly what server.py reads at import time — the check runs once at startup, not per request.
  3. Use a model from a provider you have configured if LiteLLM support is not needed.
  4. Check your docker-compose/k8s secret wiring to ensure the key reaches the container environment.

Example fix

// before (docker-compose: key missing -> 500 on litellm/ models)
// after
environment:
  - LITELLM_API_KEY=sk-1234
// then restart: docker compose up -d
Defensive patterns

Strategy: validation

Validate before calling

import os
def ensure_litellm_ready():
    if not os.environ.get("LITELLM_API_KEY"):
        raise RuntimeError("LITELLM_API_KEY is not set; litellm/ models are unavailable")

Prevention

When it happens

Trigger: POSTing to /chat/completions with data.model starting with 'litellm/' (e.g. 'litellm/gpt-4o') while the server started without the LITELLM_API_KEY env var, leaving the litellm client as None.

Common situations: Running OpenUI behind a LiteLLM proxy without exporting the key, .env file not mounted in the container, key renamed or typo'd in the environment, or deploying with secrets for other providers only.

Understand the failure class

Background: "environment variable is not set" and "Missing keys in environment" errors: what missing required env var messages mean and how to fix them — this error's family across 28 libraries.

Related errors

AI-assisted analysis of wandb/openui@42d7ab4ab6 (2026-09-01). Data as JSON: /api/errors/da5b9b7ac52eb435. Report an issue: GitHub.