{"record":{"id":"2f1a6c06f002e3b2","repo":"mlflow/mlflow","slug":"error-when-constructing-gateway-query-unsupported","errorCode":null,"errorMessage":"Error when constructing gateway query: Unsupported route type for _PromptlabModel: {route_type}","messagePattern":"Error when constructing gateway query: Unsupported route type for _PromptlabModel: (.+?)","errorType":"exception","errorClass":"MlflowException","httpStatus":null,"severity":"error","filePath":"mlflow/prompt/promptlab_model.py","lineNumber":56,"sourceCode":"            response = client.predict(\n                endpoint=self.model_route, inputs=query_data | model_parameters_as_dict\n            )\n            results.append(self._parse_gateway_response(response))\n\n        return results\n\n    def _construct_query_data(self, prompt):\n        from mlflow.deployments import MlflowDeploymentClient, get_deploy_client\n\n        client = MlflowDeploymentClient(get_deploy_client())\n        route_type = client.get_endpoint(self.model_route).endpoint_type\n\n        if route_type == \"llm/v1/completions\":\n            return {\"prompt\": prompt}\n        elif route_type == \"llm/v1/chat\":\n            return {\"messages\": [{\"content\": prompt, \"role\": \"user\"}]}\n        else:\n            raise MlflowException(\n                \"Error when constructing gateway query: \"\n                f\"Unsupported route type for _PromptlabModel: {route_type}\"\n            )\n\n    def _parse_gateway_response(self, response):\n        from mlflow.deployments import MlflowDeploymentClient, get_deploy_client\n\n        client = MlflowDeploymentClient(get_deploy_client())\n        route_type = client.get_endpoint(self.model_route).endpoint_type\n\n        if route_type == \"llm/v1/completions\":\n            return response[\"choices\"][0][\"text\"]\n        elif route_type == \"llm/v1/chat\":\n            return response[\"choices\"][0][\"message\"][\"content\"]\n        else:\n            raise MlflowException(\n                \"Error when parsing gateway response: \"\n                f\"Unsupported route type for _PromptlabModel: {route_type}\"","sourceCodeStart":38,"sourceCodeEnd":74,"githubUrl":"https://github.com/mlflow/mlflow/blob/6a27f2decc0b76eb1b54af31849784addb357dbc/mlflow/prompt/promptlab_model.py#L38-L74","documentation":"The promptlab evaluation model builds the gateway request body from the served route type. Only llm/v1/completions and llm/v1/chat are supported; if _construct_query_data receives any other route_type, it raises MlflowException indicating the route type is unsupported for _PromptlabModel.","triggerScenarios":"Deploying/invoking _PromptlabModel.predict against an AI gateway endpoint whose route_type is something other than 'llm/v1/completions' or 'llm/v1/chat' — e.g. an 'llm/v1/embeddings' route or a custom/misconfigured endpoint type returned by the gateway.","commonSituations":"Pointing prompt lab evaluation at an embeddings endpoint instead of a completions/chat one; a gateway endpoint created with a wrong or newer task type not supported by the prompt lab; reading route_type from a config with a typo (e.g. 'llm/v1/completion').","solutions":["Check the gateway endpoint's task/route type (MlflowDeploymentClient().get_endpoint or server config) and use a completions or chat endpoint","Fix typos in route type configuration so it is exactly 'llm/v1/completions' or 'llm/v1/chat'","If you need embeddings or another task type, use the appropriate evaluation model/client rather than _PromptlabModel","Upgrade MLflow if a newer route type was recently added and this model has since gained support"],"exampleFix":"// before\nendpoint = client.create_endpoint(name=\"emb\", task_type=\"llm/v1/embeddings\")  # unsupported\n\n// after\nendpoint = client.create_endpoint(name=\"chat\", task_type=\"llm/v1/chat\")","handlingStrategy":"validation","validationCode":"SUPPORTED = {\"llm/v1/completions\", \"llm/v1/chat\"}\n\ndef endpoint_is_promptlab_compatible(endpoint):\n    return getattr(endpoint, \"task_type\", None) in SUPPORTED or endpoint.get(\"task_type\") in SUPPORTED\n\n# assert endpoint_is_promptlab_compatible(ep) before predict","typeGuard":"def is_supported_route(route_type: str) -> bool:\n    return route_type in (\"llm/v1/completions\", \"llm/v1/chat\")","tryCatchPattern":"from mlflow.exceptions import MlflowException\n\ntry:\n    response = model.predict(prompt)\nexcept MlflowException as e:\n    if \"Unsupported route type\" in str(e):\n        # switch to a chat/completions gateway endpoint\n        ...","preventionTips":["Verify endpoint task_type is llm/v1/chat or llm/v1/completions before evaluation","Don't point prompt lab at embeddings endpoints","Validate route_type strings for exact spelling when configured externally","Keep MLflow updated for newly supported gateway task types"],"tags":["gateway","promptlab","route-type","llm"],"backgroundTag":"unsupported-route-type","analyzedSha":"6a27f2decc0b76eb1b54af31849784addb357dbc","analyzedAt":"2026-08-29T20:54:51.419Z","schemaVersion":2},"datasetVersion":"2026-08-29T22:17:34.462Z"}