mlflow/mlflow · error · AIGatewayException

The completions endpoint is not supported for {self.config.m

Error message

The completions endpoint is not supported for {self.config.model.name} on Vertex AI. Use the chat endpoint instead.

What it means

The Vertex AI provider deliberately rejects the completions endpoint for claude and maas model types, raising AIGatewayException with status 501 Not Implemented, because Vertex AI exposes those models only through the chat (rawPredict) API.

Source

Thrown at mlflow/gateway/providers/vertex_ai.py:297

            return self._delegate.get_endpoint_url(route_type)
        return super().get_endpoint_url(route_type)

    async def _chat(self, payload):
        if self._delegate:
            return await self._delegate._chat(payload)
        return await super()._chat(payload)

    async def _chat_stream(self, payload):
        if self._delegate:
            async for chunk in self._delegate._chat_stream(payload):
                yield chunk
            return
        async for chunk in super()._chat_stream(payload):
            yield chunk

    async def _completions(self, payload):
        if self._model_type in ("claude", "maas"):
            raise AIGatewayException(
                status_code=501,
                detail=(
                    f"The completions endpoint is not supported for {self.config.model.name} on "
                    "Vertex AI. Use the chat endpoint instead."
                ),
            )
        return await super()._completions(payload)

    async def _completions_stream(self, payload):
        if self._model_type in ("claude", "maas"):
            raise AIGatewayException(
                status_code=501,
                detail=(
                    f"The completions endpoint is not supported for {self.config.model.name} on "
                    "Vertex AI. Use the chat endpoint instead."
                ),
            )
        async for chunk in super()._completions_stream(payload):

View on GitHub (pinned to 6a27f2decc)

Solutions

1. Call the gateway's /chat/completions endpoint instead of /completions.
2. Update client code to use the chat messages format.
3. Configure a separate provider (e.g. OpenAI or Anthropic direct) if a completions endpoint is required.

Example fix

// before
client.completions.create(prompt="...", model="vertex-claude-route")
// after
client.chat.completions.create(messages=[{"role": "user", "content": "..."}], model="vertex-claude-route")

def assert_chat_only(model_type: str, endpoint: str):
    if model_type in ('claude', 'maas') and endpoint == 'completions':
        raise ValueError('Vertex AI claude/maas models support only the chat endpoint')

try:
    resp = await gateway_completions(route)
except MlflowException as e:
    if '501' in str(e) or 'not supported' in str(e):
        log.error('Use the chat/completions endpoint for Vertex AI Claude models')
        resp = await gateway_chat(route)

When it happens

Trigger: Calling the gateway's /completions endpoint (route_type llm/v1/completions) on a route whose model is a Vertex AI Claude or MaaS model.

Common situations: Client code written for OpenAI-style completions being pointed at a Vertex Claude gateway route, or migrating routes from OpenAI to Vertex AI without updating callers.

Related errors

• Unsupported route type for Vertex AI Claude: {route_type}
• Unexpected config type {config.model.config}
• Vertex AI provider requires the google-auth package. Install
• vertex_credentials must be a JSON string or path to a JSON f
• Unknown Gateway vendor: {vendor!r}

AI-assisted analysis of mlflow/mlflow@6a27f2decc (2026-08-29). Data as JSON: /api/errors/c7d0c18e847760ee. Report an issue: GitHub.