xai-org/x-algorithm · error · ValueError

resume_position is only supported in metadata mode (.valid_b

Error message

resume_position is only supported in metadata mode (.valid_batches.json)

What it means

resume_position (used to continue training from a batch id) is only implemented for metadata mode, because it relies on the .valid_batches.json ordering maintained by metadata_path. Passing resume_position with index_path (and no metadata_path) raises ValueError at construction.

Source

Thrown at phoenix/xrex/data/parquet_recsys.py:272

        interleave_k: int,
        num_kafka_partitions: int,
        skip_rows: int = 0,
        date_range: tuple[str, str] | None = None,
        continuous: bool = False,
        poll_interval_s: float = 60.0,
        resume_position: DataPosition | None = None,
        min_timestamp_ms: int | None = None,
        max_timestamp_ms: int | None = None,
        conversion_delay_columns: list[str] | None = None,
        include_action_delay_columns: bool = False,
    ):
        self._conversion_delay_columns = conversion_delay_columns
        self._include_action_delay_columns = include_action_delay_columns
        if metadata_path is None and index_path is None:
            raise ValueError("Either metadata_path or index_path must be provided")

        if resume_position is not None and metadata_path is None:
            raise ValueError(
                "resume_position is only supported in metadata mode (.valid_batches.json)"
            )

        has_time_range = min_timestamp_ms is not None or max_timestamp_ms is not None
        if has_time_range and metadata_path is None:
            raise ValueError(
                "min_timestamp_ms/max_timestamp_ms require metadata mode (.valid_batches.json)"
            )

        self._index_path = index_path
        self._metadata_path = metadata_path
        if metadata_path is not None:
            if topic_dir is None:
                topic_dir = str(Path(metadata_path).parent)
            topic_dir = os.path.abspath(topic_dir)
            self._topic_dir = topic_dir
        self._path = str(Path(index_path).parent) if index_path else topic_dir or ""
        self._batch_size = batch_size

View on GitHub (pinned to 24c60942c5)

Solutions

1. Provide metadata_path (and its .valid_batches.json) when using resume_position.
2. Or drop resume_position and restart from the beginning of the index.
3. Convert your file list into a .valid_batches.json to keep resume support.

Example fix

# before
ds = ParquetRecsysDataset(..., index_path='files.txt', resume_position=500)  # ValueError

# after
ds = ParquetRecsysDataset(..., metadata_path='topic.valid_batches.json', resume_position=500)

if resume_position is not None and metadata_path is None:
    resume_position = None  # or fail config validation
    logger.warning('resume_position ignored without metadata mode')

try:
    ds = ParquetRecsysDataset(...)
except ValueError as e:
    if 'resume_position' in str(e):
        retry_without_resume()
    else:
        raise

When it happens

Trigger: ParquetRecsysDataset(..., index_path='files.txt', resume_position=1234); resuming a job after switching from metadata mode to index mode.

Common situations: Switching deployment from .valid_batches.json to a static index while keeping resume logic; config leftovers from a previous run mode.

Related errors

• min_timestamp_ms/max_timestamp_ms require metadata mode (.va
• Either metadata_path or index_path must be provided
• Did not find any files matching {file_pattern}
• no cached cls parquet under {split_dir}
• {cls.__name__} must set TASK_GENERATOR_TYPE to be registered

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/d4130c3e16582354. Report an issue: GitHub.