mlflow/mlflow · warning · UserWarning

Searching traces without a time range constraint on UC table

Error message

Searching traces without a time range constraint on UC table locations can be slow and expensive. Consider adding a `trace.timestamp_ms` filter to your `filter_string` to limit the scan, e.g. filter_string="trace.timestamp_ms > '2024-01-01'".

What it means

When search_traces targets a Unity Catalog table location (a dotted location like catalog.schema.table), a full scan without a time filter is slow and costly because UC table traces are not indexed the same way experiment-store traces are. MLflow warns at query time when any location contains a '.' and the filter_string lacks a trace.timestamp_ms predicate.

Source

Thrown at mlflow/tracing/fluent.py:1194

                    " the `return_type='pandas'` option, or set `return_type='list'`."
                ),
            )

    _validate_list_param("locations", locations, allow_none=True)

    if flush:
        _flush_pending_async_trace_writes()

    if not experiment_ids and not locations:
        _logger.debug("Searching traces in the current active experiment")
        locations = _get_search_locations(locations)

    if (
        locations
        and any("." in loc for loc in locations)
        and (not filter_string or "trace.timestamp_ms" not in filter_string.lower())
    ):
        warnings.warn(
            "Searching traces without a time range constraint on UC table locations can be slow "
            "and expensive. Consider adding a `trace.timestamp_ms` filter to your `filter_string` "
            "to limit the scan, e.g. filter_string=\"trace.timestamp_ms > '2024-01-01'\".",
            category=UserWarning,
            stacklevel=2,
        )

    def pagination_wrapper_func(number_to_get, next_page_token):
        return TracingClient().search_traces(
            experiment_ids=experiment_ids,
            run_id=run_id,
            max_results=number_to_get,
            filter_string=filter_string,
            order_by=order_by,
            page_token=next_page_token,
            model_id=model_id,
            include_spans=include_spans,
            locations=locations,

View on GitHub (pinned to 6a27f2decc)

Solutions

Add a time bound to the filter, e.g. filter_string="trace.timestamp_ms > '2025-01-01'"
Narrow the time window as tightly as possible (and combine with other predicates like attributes.status)
If a broad scan is genuinely intended, silence the UserWarning with warnings.filterwarnings("ignore", category=UserWarning)

Example fix

// before
traces = mlflow.search_traces(locations=["main.default.traces"])
// after
traces = mlflow.search_traces(
    locations=["main.default.traces"],
    filter_string="trace.timestamp_ms > '2025-06-01'",
)

Defensive patterns

Strategy: validation

Validate before calling

import datetime
def bounded_trace_filter(filter_string: str | None, lookback_days: int = 30) -> str:
    if filter_string and "trace.timestamp_ms" in filter_string.lower():
        return filter_string
    cutoff = datetime.datetime.now(datetime.timezone.utc) - datetime.timedelta(days=lookback_days)
    ts = str(int(cutoff.timestamp() * 1000))
    base = f"trace.timestamp_ms > '{ts}'"
    return f"{filter_string} AND {base}" if filter_string else base

Type guard

def is_unbounded_uc_query(locations: list[str], filter_string: str | None) -> bool:
    return any("." in loc for loc in locations) and (
        not filter_string or "trace.timestamp_ms" not in filter_string.lower()
    )

Prevention

Always include a trace.timestamp_ms predicate when querying UC table locations
Default to a rolling lookback window in dashboards/eval jobs
Test queries against the warehouse with EXPLAIN or a small window before scheduling them

When it happens

Trigger: mlflow.search_traces(locations=['catalog.schema.trace_table']) with no filter_string, or with a filter_string that does not mention trace.timestamp_ms.

Common situations: Databricks users querying UC-managed trace tables for dashboards or evals over all history; especially painful on large tables where the query bills warehouse compute for a full table scan.

Related errors

AI-assisted analysis of mlflow/mlflow@6a27f2decc (2026-08-29). Data as JSON: /api/errors/518692a666701b5e. Report an issue: GitHub.