xai-org/x-algorithm · error · ValueError

Either metadata_path or index_path must be provided

Error message

Either metadata_path or index_path must be provided

What it means

The ParquetRecsys dataset constructor requires exactly one of metadata_path (.valid_batches.json mode) or index_path (plain file-list mode) to locate batch files. If both are None the dataset has nothing to enumerate and raises ValueError immediately.

Source

Thrown at phoenix/xrex/data/parquet_recsys.py:269

        batch_size: int,
        num_shards: int,
        shard_index: int,
        interleave_k: int,
        num_kafka_partitions: int,
        skip_rows: int = 0,
        date_range: tuple[str, str] | None = None,
        continuous: bool = False,
        poll_interval_s: float = 60.0,
        resume_position: DataPosition | None = None,
        min_timestamp_ms: int | None = None,
        max_timestamp_ms: int | None = None,
        conversion_delay_columns: list[str] | None = None,
        include_action_delay_columns: bool = False,
    ):
        self._conversion_delay_columns = conversion_delay_columns
        self._include_action_delay_columns = include_action_delay_columns
        if metadata_path is None and index_path is None:
            raise ValueError("Either metadata_path or index_path must be provided")

        if resume_position is not None and metadata_path is None:
            raise ValueError(
                "resume_position is only supported in metadata mode (.valid_batches.json)"
            )

        has_time_range = min_timestamp_ms is not None or max_timestamp_ms is not None
        if has_time_range and metadata_path is None:
            raise ValueError(
                "min_timestamp_ms/max_timestamp_ms require metadata mode (.valid_batches.json)"
            )

        self._index_path = index_path
        self._metadata_path = metadata_path
        if metadata_path is not None:
            if topic_dir is None:
                topic_dir = str(Path(metadata_path).parent)
            topic_dir = os.path.abspath(topic_dir)

View on GitHub (pinned to 24c60942c5)

Solutions

1. Pass metadata_path pointing to .valid_batches.json (enables resume/time-range features), or index_path pointing to a text file listing parquet paths.
2. Generate the index file (one absolute path per line) if you only have a directory of parquet files.
3. Verify the config key actually reaches the constructor (no typo/None default).

Example fix

# before
ds = ParquetRecsysDataset(topic_dir='/data/topic')  # ValueError

# after
open('files.txt','w').write('\n'.join(glob('/data/topic/**/*.parquet', recursive=True)))
ds = ParquetRecsysDataset(topic_dir='/data/topic', index_path='files.txt')

if metadata_path is None and index_path is None:
    index_path = generate_index_from_dir(topic_dir)  # write files.txt
assert metadata_path or index_path

try:
    ds = ParquetRecsysDataset(...)
except ValueError as e:
    raise ConfigError(str(e)) from e

When it happens

Trigger: Constructing the dataset with neither argument; passing None because a config template left both keys unset; passing an option like file_list that the constructor does not support.

Common situations: Incomplete config after copying a template; programmatic construction where the path variable is None due to an upstream if-branch.

Related errors

• resume_position is only supported in metadata mode (.valid_b
• min_timestamp_ms/max_timestamp_ms require metadata mode (.va
• Did not find any files matching {file_pattern}
• no cached cls parquet under {split_dir}
• {cls.__name__} must set TASK_GENERATOR_TYPE to be registered

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/aa7c83d4392ba323. Report an issue: GitHub.