BerriAI/litellm · error · ValueError

api_base not set. Set api_base or litellm.api_base for custo

Error message

api_base not set. Set api_base or litellm.api_base for custom endpoints

What it means

ValueError from the custom-endpoint text-completion path: it resolves url = litellm.api_base or api_base and raises when the result is None or ''. Custom (non-OpenAI) completion endpoints follow a documented POST {api_base} format, and without a URL there is nothing to POST to.

Source

Thrown at litellm/main.py:4678

def _custom_api_first_output(resp: httpx.Response | None) -> str:
    return resp.json()["data"][0]["output"][0]


def _complete_custom(ctx: _CompletionDispatchContext) -> _CompletionDispatchResult:
    api_base: Final = ctx.api_base
    headers: Final = ctx.headers
    kwargs: Final = ctx.kwargs
    max_tokens: Final = ctx.max_tokens
    messages: Final = ctx.messages
    model: Final = ctx.model
    model_response: Final = ctx.model_response
    temperature: Final = ctx.temperature
    top_p: Final = ctx.top_p

    url: Final = litellm.api_base or api_base or ""
    if url is None or url == "":
        raise ValueError("api_base not set. Set api_base or litellm.api_base for custom endpoints")

    """
    assume input to custom LLM api bases follow this format:
    resp = litellm.module_level_client.post(
        api_base,
        json={
            'model': 'meta-llama/Llama-2-13b-hf', # model name
            'params': {
                'prompt': ["The capital of France is P"],
                'max_tokens': 32,
                'temperature': 0.7,
                'top_p': 1.0,
                'top_k': 40,
            }
        }
    )

    """

View on GitHub (pinned to 77b7c6c40c)

Solutions

Pass api_base='http://localhost:8000/v1' (your server URL) on the call
Or set litellm.api_base = 'http://localhost:8000/v1' globally before invoking
Do not rely on OPENAI_API_BASE/OPENAI_BASE_URL here -- this path reads only the arg and the litellm global
Double-check the value is a non-empty string (not None/'') after config loading

Example fix

# before
resp = litellm.text_completion(model='text-completion-openai/llama-3', prompt='hi')  # ValueError

# after
resp = litellm.text_completion(model='text-completion-openai/llama-3', prompt='hi', api_base='http://localhost:8000/v1')

Defensive patterns

Strategy: validation

Validate before calling

url = litellm.api_base or api_base
if not url:
    raise SystemExit('custom endpoint needs api_base or litellm.api_base')

Type guard

def custom_base_set(url: str | None) -> bool:
    return isinstance(url, str) and bool(url.strip())

Try / catch

try:
    resp = litellm.text_completion(model='text-completion-openai/llama-3', prompt='hi', api_base=base)
except ValueError as e:
    if 'api_base not set' in str(e):
        raise RuntimeError('custom endpoint URL missing: set api_base (env vars are not read here)') from e
    raise

Prevention

This path ignores OPENAI_API_BASE/OPENAI_BASE_URL -- always pass api_base or set litellm.api_base explicitly
Centralize your local-LLM base URL in one config constant reused by every call
After moving a local server, grep for the old host:port across config, not just env files
Assert the base URL is a non-empty http(s) string before dispatch in shared helper code

When it happens

Trigger: text_completion / completion on a custom openai-compatible provider (e.g. model='text-completion-openai/<name>' or a custom handler) where neither the api_base argument nor litellm.api_base was set (note: environment fallbacks are not consulted on this path).

Common situations: Relying on OPENAI_API_BASE env var, which this custom path ignores; setting api_base on a different call than the one that dispatches; local LLM server (llama.cpp, vllm) moved ports and the global was never updated.

Related errors

AI-assisted analysis of BerriAI/litellm@77b7c6c40c (2026-08-18). Data as JSON: /api/errors/b7e374e3b045b0a3. Report an issue: GitHub.