mlflow/mlflow · error · MlflowException

Failed to perform one or more operations on the run with ID

Error message

Failed to perform one or more operations on the run with ID {run_id}. Failed operations: {failures}

What it means

MlflowAutologgingQueue._flush_pending_operations performs deferred run operations (log metrics/params, set_terminated) for the run managed by the autologging client; any returned Exception results are collected into `failures` and re-raised as this aggregated MlflowException naming the run ID. It indicates the autologging session could not fully persist run data for that run.

Source

Thrown at mlflow/utils/autologging_utils/client.py:392

            self._try_operation(self._client.log_inputs, run_id=run_id, datasets=datasets_batch)
            for datasets_batch in chunk_list(
                pending_operations.datasets_queue, chunk_size=MAX_DATASETS_PER_BATCH
            )
        )

        if pending_operations.set_terminated:
            operation_results.append(
                self._try_operation(
                    self._client.set_terminated,
                    run_id=run_id,
                    status=pending_operations.set_terminated.status,
                    end_time=pending_operations.set_terminated.end_time,
                )
            )

        failures = [result for result in operation_results if isinstance(result, Exception)]
        if len(failures) > 0:
            raise MlflowException(
                message=(
                    f"Failed to perform one or more operations on the run with ID {run_id}."
                    f" Failed operations: {failures}"
                )
            )


class _PendingRunOperations:
    """
    Represents a collection of queued / pending MLflow Run operations.
    """

    def __init__(self, run_id):
        self.run_id = run_id
        self.create_run = None
        self.set_terminated = None
        self.params_queue = []
        self.tags_queue = []

View on GitHub (pinned to 6a27f2decc)

Solutions

Use the run_id in the message to query the run and confirm it still exists (mlflow.get_run(run_id))
Inspect the individual failures list for the underlying exception per operation
Check param/metric names and values against MLflow length limits and shorten offending keys
Re-authenticate (Databricks/hosted tracking) if the cause is an auth error
Retry the failed operations manually with mlflow.log_param/log_metric/set_terminated

Example fix

// before
mlflow.sklearn.autolog()
model.fit(X, y)  # raises: Failed operations: [MlflowException(...)]
// after
run = mlflow.active_run()
try:
    mlflow.set_terminated(run.info.run_id)
except MlflowException as e:
    print(run.info.run_id, e)  # inspect per-operation failures and retry selectively

Defensive patterns

Strategy: try-catch

Validate before calling

from mlflow.tracking import MlflowClient
# confirm the autologged run exists before flushing pending ops
MlflowClient().get_run(run_id)

Type guard

def is_exception_result(result):
    return isinstance(result, Exception)

Try / catch

try:
    client._flush_pending_operations(run_id)
except MlflowException as e:
    logger.error('failed ops for run %s: %s', run_id, e)
    # retry individual mlflow.log_param/log_metric calls synchronously

Prevention

Don't delete runs while an autologging session is open
Keep param names <=250 chars and values <=6000 chars
Refresh auth tokens for long Databricks/hosted training jobs
Check the per-operation failures list to target retries precisely

When it happens

Trigger: Ending an autologged run (or error_termination/flush path) when deferred operations against run {run_id} throw — e.g. the run was already deleted, the tracking store rejected a param/metric (name/value too long), or authentication expired mid-session.

Common situations: Run deleted or store cleaned while autologging session still open; param key exceeding MLflow's 250-char / value 6000-char limits; token expiry with Databricks tracking during long training jobs.

Related errors

AI-assisted analysis of mlflow/mlflow@6a27f2decc (2026-08-29). Data as JSON: /api/errors/bef6cc6605ccb20b. Report an issue: GitHub.