mlflow/mlflow · warning · UserWarning

MLFLOW_DEPLOYMENT_PREDICT_TOTAL_TIMEOUT ({retry_timeout_seco

Error message

MLFLOW_DEPLOYMENT_PREDICT_TOTAL_TIMEOUT ({retry_timeout_seconds}s) is set lower than MLFLOW_DEPLOYMENT_PREDICT_TIMEOUT ({timeout}s). This means the total retry timeout could expire before a single request completes, causing premature failures. For long-running predictions, ensure MLFLOW_DEPLOYMENT_PREDICT_TOTAL_TIMEOUT >= MLFLOW_DEPLOYMENT_PREDICT_TIMEOUT. Recommended: Set MLFLOW_DEPLOYMENT_PREDICT_TOTAL_TIMEOUT to at least {timeout}s.

What it means

MLflow's REST client warns when MLFLOW_DEPLOYMENT_PREDICT_TOTAL_TIMEOUT (aggregate retry budget) is smaller than MLFLOW_DEPLOYMENT_PREDICT_TIMEOUT (per-request timeout). In that configuration the total budget can expire before even one request finishes, so retries are useless and requests fail prematurely.

Source

Thrown at mlflow/utils/rest_utils.py:470

    if backoff_factor < 0:
        raise MlflowException(
            message="The backoff_factor value must be either 0 a positive integer. "
            f"Got {backoff_factor}",
            error_code=INVALID_PARAMETER_VALUE,
        )


def validate_deployment_timeout_config(timeout: int | None, retry_timeout_seconds: int | None):
    """
    Validate that total retry timeout is not less than single request timeout.

    Args:
        timeout: Maximum time for a single HTTP request (in seconds)
        retry_timeout_seconds: Maximum time for all retry attempts combined (in seconds)
    """
    if timeout is not None and retry_timeout_seconds is not None:
        if retry_timeout_seconds < timeout:
            warnings.warn(
                f"MLFLOW_DEPLOYMENT_PREDICT_TOTAL_TIMEOUT ({retry_timeout_seconds}s) is set "
                f"lower than MLFLOW_DEPLOYMENT_PREDICT_TIMEOUT ({timeout}s). This means the "
                "total retry timeout could expire before a single request completes, causing "
                "premature failures. For long-running predictions, ensure "
                "MLFLOW_DEPLOYMENT_PREDICT_TOTAL_TIMEOUT >= MLFLOW_DEPLOYMENT_PREDICT_TIMEOUT. "
                f"Recommended: Set MLFLOW_DEPLOYMENT_PREDICT_TOTAL_TIMEOUT to at least {timeout}s.",
                stacklevel=2,
            )


def _time_sleep(seconds: float) -> None:
    """
    This function is specifically mocked in `test_rest_utils.py` to test the backoff logic in
    isolation. We avoid wrapping `time.sleep` globally to prevent interfering with unrelated sleep
    calls elsewhere in the codebase or in third-party libraries.
    """
    time.sleep(seconds)

View on GitHub (pinned to 6a27f2decc)

Solutions

Set MLFLOW_DEPLOYMENT_PREDICT_TOTAL_TIMEOUT >= MLFLOW_DEPLOYMENT_PREDICT_TIMEOUT (recommended: at least the per-request value)
Lower MLFLOW_DEPLOYMENT_PREDICT_TIMEOUT if you want fast failure per request
Increase the total timeout for long-running predictions, e.g. MLFLOW_DEPLOYMENT_PREDICT_TOTAL_TIMEOUT=3600
Unset one of the variables so MLflow applies its defaults

Example fix

// before
export MLFLOW_DEPLOYMENT_PREDICT_TIMEOUT=120
export MLFLOW_DEPLOYMENT_PREDICT_TOTAL_TIMEOUT=60
// after
export MLFLOW_DEPLOYMENT_PREDICT_TIMEOUT=120
export MLFLOW_DEPLOYMENT_PREDICT_TOTAL_TIMEOUT=600

Defensive patterns

Strategy: validation

Validate before calling

import os
t = os.environ.get("MLFLOW_DEPLOYMENT_PREDICT_TIMEOUT")
tt = os.environ.get("MLFLOW_DEPLOYMENT_PREDICT_TOTAL_TIMEOUT")
if t and tt and float(tt) < float(t):
    raise ValueError("MLFLOW_DEPLOYMENT_PREDICT_TOTAL_TIMEOUT must be >= MLFLOW_DEPLOYMENT_PREDICT_TIMEOUT")

Try / catch

import warnings
with warnings.catch_warnings():
    warnings.filterwarnings("error", message="MLFLOW_DEPLOYMENT_PREDICT_TOTAL_TIMEOUT")
    try:
        client.predict(deployment_id, df)
    except UserWarning as w:
        print("fix timeout config:", w)

Prevention

Always set TOTAL >= per-request timeout in deployment configs
For long-running predictions, set generous total timeouts (e.g. 3600s)
Add a startup config check in services using the deployment client

When it happens

Trigger: Calling deployment client predict/_call_endpoint (or streaming) with MLFLOW_DEPLOYMENT_PREDICT_TOTAL_TIMEOUT < MLFLOW_DEPLOYMENT_PREDICT_TIMEOUT, via validate_deployment_timeout_config.

Common situations: Users setting a low total timeout to 'fail fast' without realizing the per-request timeout must be smaller; long-running model predictions (minutes) with default single-request timeout higher than a short total timeout.

Understand the failure class

Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

AI-assisted analysis of mlflow/mlflow@6a27f2decc (2026-08-29). Data as JSON: /api/errors/ad9018265b2a732c. Report an issue: GitHub.