BerriAI/litellm · error · ValueError

api_base must be provided for Hosted VLLM rerank

Error message

api_base must be provided for Hosted VLLM rerank

What it means

Raised by HostedVLLM TranscriptionConfig.get_complete_url when api_base is falsy for an audio transcription call. The vLLM-compatible server URL is required because hosted_vllm is self-hosted. Note the message text ('...must be provided for Hosted VLLM rerank') is a copy-paste from the rerank handler — it is misleading, but the actual failure is the missing api_base for transcriptions.

Source

Thrown at litellm/llms/hosted_vllm/transcriptions/transformation.py:48

    def __init__(self) -> None:
        pass

    def get_complete_url(
        self,
        api_base: str | None,
        api_key: str | None,
        model: str,
        optional_params: dict,
        litellm_params: dict,
        stream: bool | None = None,
    ) -> str:
        if api_base:
            # Remove trailing slashes and ensure clean base URL
            api_base = api_base.rstrip("/")
            if not api_base.endswith("/v1/audio/transcriptions"):
                api_base = f"{api_base}/v1/audio/transcriptions"
            return api_base
        raise ValueError("api_base must be provided for Hosted VLLM rerank")

    def transform_audio_transcription_request(
        self,
        model: str,
        audio_file: FileTypes,
        optional_params: dict,
        litellm_params: dict,
    ) -> AudioTranscriptionRequestData:
        """
        Transform the audio transcription request
        """

        data: Final = {"model": model, "file": audio_file, **optional_params}

        return AudioTranscriptionRequestData(
            data=data,
        )

View on GitHub (pinned to 6c2dcb801b)

Solutions

1. Pass api_base to the transcription call: litellm.transcription(model='hosted_vllm/...', file=f, api_base='http://vllm-host:8000').
2. Ignore the word 'rerank' in the message — for transcription calls this simply means api_base was empty.
3. Ensure the vLLM server actually exposes /v1/audio/transcriptions (path appended automatically) with an audio model loaded.

Example fix

# before
litellm.transcription(model='hosted_vllm/whisper-large-v3', file=audio_file)
# raises ValueError: 'api_base must be provided for Hosted VLLM rerank'  (misleading text; it's the transcription api_base)

# after
litellm.transcription(
    model='hosted_vllm/whisper-large-v3',
    file=audio_file,
    api_base='http://localhost:8000',
)

def vllm_transcription_base(api_base: str | None) -> str:
    if not api_base:
        raise ValueError("hosted_vllm transcription requires an explicit api_base")
    return api_base

try:
    litellm.transcription(model="hosted_vllm/...", file=f, api_base=base)
except ValueError as e:
    if "api_base must be provided" in str(e):
        # note: message text says 'rerank' but this is the transcription handler
        raise RuntimeError("missing api_base for hosted_vllm transcription") from e
    raise

When it happens

Trigger: Calling litellm.transcription(model='hosted_vllm/whisper-...', file=audio_file) without api_base and without a resolvable default. Like the rerank handler, this method only checks the explicit api_base argument (no env-var fallback in this method).

Common situations: New audio-transcription feature added to an existing app where only chat calls passed api_base; assuming HOSTED_VLLM_API_BASE covers transcription (it does not in this handler); the confusing 'rerank' wording sending developers to debug their rerank config instead of supplying the transcription URL.

Related errors

• api_base is required for hosted_vllm embeddings
• api_base must be provided for Hosted VLLM rerank
• api_base not set for Hosted VLLM responses API. Set via api_
• API base is required for OpenAI image variations
• API base is required for Topaz image variations

AI-assisted analysis of BerriAI/litellm@6c2dcb801b (2026-08-15). Data as JSON: /api/errors/72aa26fd9fbe57b3. Report an issue: GitHub.