mlflow/mlflow · error · MlflowException

NOT_FOUND

NOT_FOUND

Error message

Trace with ID {trace_id} is not found.

What it means

`MlflowTracingClient.get_trace` polls for a trace whose info is not yet visible, retrying with a sleep interval. If the trace still cannot be found after retries, it raises NOT_FOUND. This is the terminal 'trace does not exist (yet)' signal for the given trace_id.

Source

Thrown at mlflow/tracing/client.py:220

            initial_interval = max(0.0, MLFLOW_GET_TRACE_OTEL_INITIAL_RETRY_INTERVAL_SECONDS.get())
            max_interval = max(0.0, MLFLOW_GET_TRACE_OTEL_MAX_RETRY_INTERVAL_SECONDS.get())
            attempt = 0
            while True:
                if traces := self.store.batch_get_traces([trace_id], location):
                    return traces[0]

                remaining = deadline - time.monotonic()
                if remaining <= 0:
                    break

                interval = min(initial_interval * 2**attempt, max_interval, remaining)
                attempt += 1
                _logger.debug(
                    f"Trace not found, retrying in {interval:.2f} seconds (attempt {attempt})"
                )
                time.sleep(interval)

            raise MlflowException(
                message=f"Trace with ID {trace_id} is not found.",
                error_code=NOT_FOUND,
            )
        else:
            try:
                trace_info = self.get_trace_info(trace_id)
                # if the trace is stored in the tracking store or archive repo, load spans via the
                # store/server path; otherwise, load spans from the artifact repository
                if trace_info.tags.get(TraceTagKey.SPANS_LOCATION) in (
                    SpansLocation.TRACKING_STORE,
                    SpansLocation.ARCHIVE_REPO,
                ):
                    try:
                        return self.store.get_trace(trace_id)
                    except MlflowNotImplementedException:
                        pass
                    if traces := self.store.batch_get_traces([trace_info.trace_id]):
                        return traces[0]

View on GitHub (pinned to 6a27f2decc)

Solutions

1. Verify the trace_id is complete and from the current tracking URI/experiment.
2. Wait longer or ensure the producer finished logging traces before reading (or retry with backoff in your code).
3. List recent traces with MlflowTracingClient().search_traces(...) to confirm the ID exists.

Example fix

// before
trace = client.get_trace(trace_id)  # raises NOT_FOUND right after async log

// after
import time
for _ in range(5):
    try:
        trace = client.get_trace(trace_id)
        break
    except MlflowException as e:
        if e.error_code != "NOT_FOUND":
            raise
        time.sleep(2)

# pre-check existence
infos = client.search_traces(experiment_locations=[exp_id], max_results=100)
exists = any(t.trace_id == trace_id for t in infos)

import time
for attempt in range(5):
    try:
        return client.get_trace(trace_id)
    except MlflowException as e:
        if e.error_code != "NOT_FOUND":
            raise
        time.sleep(2 ** attempt)

When it happens

Trigger: Calling get_trace (directly or via get_trace_info/set_trace_tag/delete_trace_tag/log_assessment wrappers) with a trace_id that does not exist in the backing store, or one whose info is not visible within the retry window (e.g. async trace creation still in flight).

Common situations: Querying a trace immediately after an async logging call before the backend persists it; truncated/mistyped trace ID; wrong tracking server or experiment backend; trace was deleted.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• INVALID_PARAMETER_VALUE
• INVALID_PARAMETER_VALUE
• trace_id is required but was empty
• span_id is required but was empty
• TraceData.from_dict() expects a dictionary. Got: {type(d).__

AI-assisted analysis of mlflow/mlflow@6a27f2decc (2026-08-29). Data as JSON: /api/errors/04395c3e627b27df. Report an issue: GitHub.