xai-org/x-algorithm · error · ValueError

min_timestamp_ms/max_timestamp_ms require metadata mode (.va

Error message

min_timestamp_ms/max_timestamp_ms require metadata mode (.valid_batches.json)

What it means

Time-range filtering (min_timestamp_ms / max_timestamp_ms) resolves batch ranges via the valid-batches metadata, so it requires metadata_path. Supplying a time range together with only index_path raises ValueError at construction, since the index file carries no timestamp information.

Source

Thrown at phoenix/xrex/data/parquet_recsys.py:278

        resume_position: DataPosition | None = None,
        min_timestamp_ms: int | None = None,
        max_timestamp_ms: int | None = None,
        conversion_delay_columns: list[str] | None = None,
        include_action_delay_columns: bool = False,
    ):
        self._conversion_delay_columns = conversion_delay_columns
        self._include_action_delay_columns = include_action_delay_columns
        if metadata_path is None and index_path is None:
            raise ValueError("Either metadata_path or index_path must be provided")

        if resume_position is not None and metadata_path is None:
            raise ValueError(
                "resume_position is only supported in metadata mode (.valid_batches.json)"
            )

        has_time_range = min_timestamp_ms is not None or max_timestamp_ms is not None
        if has_time_range and metadata_path is None:
            raise ValueError(
                "min_timestamp_ms/max_timestamp_ms require metadata mode (.valid_batches.json)"
            )

        self._index_path = index_path
        self._metadata_path = metadata_path
        if metadata_path is not None:
            if topic_dir is None:
                topic_dir = str(Path(metadata_path).parent)
            topic_dir = os.path.abspath(topic_dir)
            self._topic_dir = topic_dir
        self._path = str(Path(index_path).parent) if index_path else topic_dir or ""
        self._batch_size = batch_size
        self._num_shards = num_shards
        self._shard_index = shard_index
        self._interleave_k = interleave_k
        self._date_range = date_range
        self._continuous = continuous
        self._poll_interval_s = poll_interval_s

View on GitHub (pinned to 24c60942c5)

Solutions

Use metadata_path with a .valid_batches.json that supports time-range resolution.
Or pre-filter the index file yourself to the desired date range (the index path supports _date_range filtering).
Remove min/max_timestamp_ms keys when in index mode.

Example fix

# before
ds = ParquetRecsysDataset(..., index_path='files.txt', min_timestamp_ms=t0)  # ValueError

# after
ds = ParquetRecsysDataset(..., metadata_path='topic.valid_batches.json', min_timestamp_ms=t0)

Defensive patterns

Strategy: validation

Validate before calling

if (min_timestamp_ms or max_timestamp_ms) and metadata_path is None:
    use_date_range_on_partitions_instead()  # e.g. date_range=('2026-08-01','2026-08-28')

Try / catch

try:
    ds = ParquetRecsysDataset(...)
except ValueError as e:
    if 'timestamp' in str(e):
        switch_to_partition_date_range()
    else:
        raise

Prevention

Prefer partition date_range for coarse filtering on index mode.
Keep .valid_batches.json available when timestamp filtering is required.

When it happens

Trigger: ParquetRecsysDataset(..., index_path='files.txt', min_timestamp_ms=1700000000000); adding a backfill window to a config that uses index mode.

Common situations: Attempting incremental training windows on a plain index listing; migrating configs between modes without removing time-range keys.

Related errors

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/bb65457983dc3525. Report an issue: GitHub.