mlflow/mlflow · error · ValueError

One or more lists in the returned prediction response are em

Error message

One or more lists in the returned prediction response are empty

What it means

EmbeddingsResponse requires each element of predictions to be a non-empty embedding vector. If every inner list is empty, there are no usable embeddings, so the validator raises this ValueError, which becomes a pydantic ValidationError and then a 502 AIGatewayException.

Source

Thrown at mlflow/gateway/providers/mlflow.py:45

                predictions = next(iter(predictions.values()))
            else:
                predictions = predictions.get("choices", predictions)
            if not predictions:
                raise ValueError("The input list is empty")
        return predictions


class EmbeddingsResponse(BaseModel):
    predictions: list[list[StrictFloat]]

    @field_validator("predictions", mode="before")
    def validate_predictions(cls, predictions):
        if isinstance(predictions, list) and not predictions:
            raise ValueError("The input list is empty")
        if isinstance(predictions, list) and all(
            isinstance(item, list) and not item for item in predictions
        ):
            raise ValueError("One or more lists in the returned prediction response are empty")
        elif all(isinstance(item, float) for item in predictions):
            return [predictions]
        else:
            return predictions


class MlflowModelServingProvider(BaseProvider):
    DISPLAY_NAME = "MLflow Model Serving"
    CONFIG_TYPE = MlflowModelServingConfig

    def __init__(self, config: EndpointConfig, enable_tracing: bool = False) -> None:
        super().__init__(config, enable_tracing=enable_tracing)
        if config.model.config is None or not isinstance(
            config.model.config, MlflowModelServingConfig
        ):
            raise TypeError(f"Invalid config type {config.model.config}")
        self.mlflow_config: MlflowModelServingConfig = config.model.config
        self.headers = {"Content-Type": "application/json"}

View on GitHub (pinned to 6a27f2decc)

Solutions

1. Fix the embedding model/serving code so each prediction contains a real vector
2. Check tokenization/truncation settings that could zero out vectors
3. Handle the 502 error client-side and validate the serving endpoint's output shape

Example fix

// before
{"predictions": [[], []]}
// after
{"predictions": [[0.1, 0.2], [0.3, 0.4]]}

preds = resp.json().get("predictions")
if isinstance(preds, list) and preds and all(isinstance(v, list) and not v for v in preds):
    raise ValueError("all embedding vectors are empty")

def has_nonempty_vectors(resp: dict) -> bool:
    p = resp.get("predictions")
    return isinstance(p, list) and any(isinstance(v, list) and len(v) > 0 for v in p)

try:
    emb = client.embeddings(route, payload)
except AIGatewayException as e:
    if "empty" in str(e.detail):
        # investigate the serving model's vector output
        ...

When it happens

Trigger: Model returns {"predictions": [[], []]} (or any list of empty lists) — every inner list is empty, matching the all(...) condition.

Common situations: Embedding model silently truncating output; serving pipeline producing empty arrays per row; tokenizer errors yielding zero tokens per input.

Related errors

• The input list is empty
• The dict format is invalid for this route type. Ensure the s
• The gateway configuration is invalid: {e}
• Invalid parameter {k2}. Use {k1} instead.
• Wrong type for logprobs. It should be an 32bit integer.

AI-assisted analysis of mlflow/mlflow@6a27f2decc (2026-08-29). Data as JSON: /api/errors/38f4f51e736ab4a9. Report an issue: GitHub.