{"record":{"id":"bb65457983dc3525","repo":"xai-org/x-algorithm","slug":"min-timestamp-ms-max-timestamp-ms-require-metadata","errorCode":null,"errorMessage":"min_timestamp_ms/max_timestamp_ms require metadata mode (.valid_batches.json)","messagePattern":"min_timestamp_ms/max_timestamp_ms require metadata mode \\(\\.valid_batches\\.json\\)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"phoenix/xrex/data/parquet_recsys.py","lineNumber":278,"sourceCode":"        resume_position: DataPosition | None = None,\n        min_timestamp_ms: int | None = None,\n        max_timestamp_ms: int | None = None,\n        conversion_delay_columns: list[str] | None = None,\n        include_action_delay_columns: bool = False,\n    ):\n        self._conversion_delay_columns = conversion_delay_columns\n        self._include_action_delay_columns = include_action_delay_columns\n        if metadata_path is None and index_path is None:\n            raise ValueError(\"Either metadata_path or index_path must be provided\")\n\n        if resume_position is not None and metadata_path is None:\n            raise ValueError(\n                \"resume_position is only supported in metadata mode (.valid_batches.json)\"\n            )\n\n        has_time_range = min_timestamp_ms is not None or max_timestamp_ms is not None\n        if has_time_range and metadata_path is None:\n            raise ValueError(\n                \"min_timestamp_ms/max_timestamp_ms require metadata mode (.valid_batches.json)\"\n            )\n\n        self._index_path = index_path\n        self._metadata_path = metadata_path\n        if metadata_path is not None:\n            if topic_dir is None:\n                topic_dir = str(Path(metadata_path).parent)\n            topic_dir = os.path.abspath(topic_dir)\n            self._topic_dir = topic_dir\n        self._path = str(Path(index_path).parent) if index_path else topic_dir or \"\"\n        self._batch_size = batch_size\n        self._num_shards = num_shards\n        self._shard_index = shard_index\n        self._interleave_k = interleave_k\n        self._date_range = date_range\n        self._continuous = continuous\n        self._poll_interval_s = poll_interval_s","sourceCodeStart":260,"sourceCodeEnd":296,"githubUrl":"https://github.com/xai-org/x-algorithm/blob/24c60942c5c5fdad3a6addffb4c6e6d2f228f04f/phoenix/xrex/data/parquet_recsys.py#L260-L296","documentation":"Time-range filtering (min_timestamp_ms / max_timestamp_ms) resolves batch ranges via the valid-batches metadata, so it requires metadata_path. Supplying a time range together with only index_path raises ValueError at construction, since the index file carries no timestamp information.","triggerScenarios":"ParquetRecsysDataset(..., index_path='files.txt', min_timestamp_ms=1700000000000); adding a backfill window to a config that uses index mode.","commonSituations":"Attempting incremental training windows on a plain index listing; migrating configs between modes without removing time-range keys.","solutions":["Use metadata_path with a .valid_batches.json that supports time-range resolution.","Or pre-filter the index file yourself to the desired date range (the index path supports _date_range filtering).","Remove min/max_timestamp_ms keys when in index mode."],"exampleFix":"# before\nds = ParquetRecsysDataset(..., index_path='files.txt', min_timestamp_ms=t0)  # ValueError\n\n# after\nds = ParquetRecsysDataset(..., metadata_path='topic.valid_batches.json', min_timestamp_ms=t0)","handlingStrategy":"validation","validationCode":"if (min_timestamp_ms or max_timestamp_ms) and metadata_path is None:\n    use_date_range_on_partitions_instead()  # e.g. date_range=('2026-08-01','2026-08-28')","typeGuard":null,"tryCatchPattern":"try:\n    ds = ParquetRecsysDataset(...)\nexcept ValueError as e:\n    if 'timestamp' in str(e):\n        switch_to_partition_date_range()\n    else:\n        raise","preventionTips":["Prefer partition date_range for coarse filtering on index mode.","Keep .valid_batches.json available when timestamp filtering is required."],"tags":["configuration","time-range","dataset","argument-combination"],"backgroundTag":"invalid-argument-combination","analyzedSha":"24c60942c5c5fdad3a6addffb4c6e6d2f228f04f","analyzedAt":"2026-08-28T11:40:14.686Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}