mlflow/mlflow · error · MlflowException

Error when constructing gateway query: Unsupported route typ

Error message

Error when constructing gateway query: Unsupported route type for _PromptlabModel: {route_type}

What it means

The promptlab evaluation model builds the gateway request body from the served route type. Only llm/v1/completions and llm/v1/chat are supported; if _construct_query_data receives any other route_type, it raises MlflowException indicating the route type is unsupported for _PromptlabModel.

Source

Thrown at mlflow/prompt/promptlab_model.py:56

            response = client.predict(
                endpoint=self.model_route, inputs=query_data | model_parameters_as_dict
            )
            results.append(self._parse_gateway_response(response))

        return results

    def _construct_query_data(self, prompt):
        from mlflow.deployments import MlflowDeploymentClient, get_deploy_client

        client = MlflowDeploymentClient(get_deploy_client())
        route_type = client.get_endpoint(self.model_route).endpoint_type

        if route_type == "llm/v1/completions":
            return {"prompt": prompt}
        elif route_type == "llm/v1/chat":
            return {"messages": [{"content": prompt, "role": "user"}]}
        else:
            raise MlflowException(
                "Error when constructing gateway query: "
                f"Unsupported route type for _PromptlabModel: {route_type}"
            )

    def _parse_gateway_response(self, response):
        from mlflow.deployments import MlflowDeploymentClient, get_deploy_client

        client = MlflowDeploymentClient(get_deploy_client())
        route_type = client.get_endpoint(self.model_route).endpoint_type

        if route_type == "llm/v1/completions":
            return response["choices"][0]["text"]
        elif route_type == "llm/v1/chat":
            return response["choices"][0]["message"]["content"]
        else:
            raise MlflowException(
                "Error when parsing gateway response: "
                f"Unsupported route type for _PromptlabModel: {route_type}"

View on GitHub (pinned to 6a27f2decc)

Solutions

1. Check the gateway endpoint's task/route type (MlflowDeploymentClient().get_endpoint or server config) and use a completions or chat endpoint
2. Fix typos in route type configuration so it is exactly 'llm/v1/completions' or 'llm/v1/chat'
3. If you need embeddings or another task type, use the appropriate evaluation model/client rather than _PromptlabModel
4. Upgrade MLflow if a newer route type was recently added and this model has since gained support

Example fix

// before
endpoint = client.create_endpoint(name="emb", task_type="llm/v1/embeddings")  # unsupported

// after
endpoint = client.create_endpoint(name="chat", task_type="llm/v1/chat")

SUPPORTED = {"llm/v1/completions", "llm/v1/chat"}

def endpoint_is_promptlab_compatible(endpoint):
    return getattr(endpoint, "task_type", None) in SUPPORTED or endpoint.get("task_type") in SUPPORTED

# assert endpoint_is_promptlab_compatible(ep) before predict

def is_supported_route(route_type: str) -> bool:
    return route_type in ("llm/v1/completions", "llm/v1/chat")

from mlflow.exceptions import MlflowException

try:
    response = model.predict(prompt)
except MlflowException as e:
    if "Unsupported route type" in str(e):
        # switch to a chat/completions gateway endpoint
        ...

When it happens

Trigger: Deploying/invoking _PromptlabModel.predict against an AI gateway endpoint whose route_type is something other than 'llm/v1/completions' or 'llm/v1/chat' — e.g. an 'llm/v1/embeddings' route or a custom/misconfigured endpoint type returned by the gateway.

Common situations: Pointing prompt lab evaluation at an embeddings endpoint instead of a completions/chat one; a gateway endpoint created with a wrong or newer task type not supported by the prompt lab; reading route_type from a config with a typo (e.g. 'llm/v1/completion').

Related errors

• Sanitization LLM response is missing 'choices[0].message.con
• Sanitization LLM returned invalid JSON.
• Invalid route type {route_type}
• Unsupported route_type '{route_type}' for Databricks provide
• Invalid route type {route_type}

AI-assisted analysis of mlflow/mlflow@6a27f2decc (2026-08-29). Data as JSON: /api/errors/2f1a6c06f002e3b2. Report an issue: GitHub.