{"record":{"id":"38f4f51e736ab4a9","repo":"mlflow/mlflow","slug":"one-or-more-lists-in-the-returned-prediction-respo","errorCode":null,"errorMessage":"One or more lists in the returned prediction response are empty","messagePattern":"One or more lists in the returned prediction response are empty","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"mlflow/gateway/providers/mlflow.py","lineNumber":45,"sourceCode":"                predictions = next(iter(predictions.values()))\n            else:\n                predictions = predictions.get(\"choices\", predictions)\n            if not predictions:\n                raise ValueError(\"The input list is empty\")\n        return predictions\n\n\nclass EmbeddingsResponse(BaseModel):\n    predictions: list[list[StrictFloat]]\n\n    @field_validator(\"predictions\", mode=\"before\")\n    def validate_predictions(cls, predictions):\n        if isinstance(predictions, list) and not predictions:\n            raise ValueError(\"The input list is empty\")\n        if isinstance(predictions, list) and all(\n            isinstance(item, list) and not item for item in predictions\n        ):\n            raise ValueError(\"One or more lists in the returned prediction response are empty\")\n        elif all(isinstance(item, float) for item in predictions):\n            return [predictions]\n        else:\n            return predictions\n\n\nclass MlflowModelServingProvider(BaseProvider):\n    DISPLAY_NAME = \"MLflow Model Serving\"\n    CONFIG_TYPE = MlflowModelServingConfig\n\n    def __init__(self, config: EndpointConfig, enable_tracing: bool = False) -> None:\n        super().__init__(config, enable_tracing=enable_tracing)\n        if config.model.config is None or not isinstance(\n            config.model.config, MlflowModelServingConfig\n        ):\n            raise TypeError(f\"Invalid config type {config.model.config}\")\n        self.mlflow_config: MlflowModelServingConfig = config.model.config\n        self.headers = {\"Content-Type\": \"application/json\"}","sourceCodeStart":27,"sourceCodeEnd":63,"githubUrl":"https://github.com/mlflow/mlflow/blob/6a27f2decc0b76eb1b54af31849784addb357dbc/mlflow/gateway/providers/mlflow.py#L27-L63","documentation":"EmbeddingsResponse requires each element of predictions to be a non-empty embedding vector. If every inner list is empty, there are no usable embeddings, so the validator raises this ValueError, which becomes a pydantic ValidationError and then a 502 AIGatewayException.","triggerScenarios":"Model returns {\"predictions\": [[], []]} (or any list of empty lists) — every inner list is empty, matching the all(...) condition.","commonSituations":"Embedding model silently truncating output; serving pipeline producing empty arrays per row; tokenizer errors yielding zero tokens per input.","solutions":["Fix the embedding model/serving code so each prediction contains a real vector","Check tokenization/truncation settings that could zero out vectors","Handle the 502 error client-side and validate the serving endpoint's output shape"],"exampleFix":"// before\n{\"predictions\": [[], []]}\n// after\n{\"predictions\": [[0.1, 0.2], [0.3, 0.4]]}","handlingStrategy":"validation","validationCode":"preds = resp.json().get(\"predictions\")\nif isinstance(preds, list) and preds and all(isinstance(v, list) and not v for v in preds):\n    raise ValueError(\"all embedding vectors are empty\")","typeGuard":"def has_nonempty_vectors(resp: dict) -> bool:\n    p = resp.get(\"predictions\")\n    return isinstance(p, list) and any(isinstance(v, list) and len(v) > 0 for v in p)","tryCatchPattern":"try:\n    emb = client.embeddings(route, payload)\nexcept AIGatewayException as e:\n    if \"empty\" in str(e.detail):\n        # investigate the serving model's vector output\n        ...","preventionTips":["Check tokenizer/truncation settings that can zero out vectors","Add a serving-side assertion that vectors are non-empty","Monitor embedding dimensions for anomalies"],"tags":["pydantic","validation","embeddings","gateway"],"backgroundTag":"empty-predictions-response","analyzedSha":"6a27f2decc0b76eb1b54af31849784addb357dbc","analyzedAt":"2026-08-29T20:54:51.419Z","schemaVersion":2},"datasetVersion":"2026-08-29T22:17:34.462Z"}