{"record":{"id":"71e43516d68a1f9e","repo":"xai-org/x-algorithm","slug":"index-file-index-path-not-found","errorCode":null,"errorMessage":"Index file {index_path} not found","messagePattern":"Index file (.+?) not found","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"phoenix/xrex/data/parquet_recsys.py","lineNumber":432,"sourceCode":"        if start > max_batch:\n            return []\n\n        ready: list[list[str]] = []\n        for bid in range(start, max_batch + 1):\n            my_files = [\n                _batch_path(self._topic_dir, p, bid)\n                for p in range(num_partitions)\n                if p % self._num_shards == self._shard_index\n            ]\n            ready.append(my_files)\n            self._next_batch_id = bid + 1\n\n        return ready\n\n    def _get_ready_batches_from_index(self) -> list[list[str]]:\n        index_path = self._index_path\n        if index_path is None or not os.path.isfile(index_path):\n            raise ValueError(f\"Index file {index_path} not found\")\n\n        with open(index_path) as f:\n            all_files = [line.strip() for line in f if line.strip()]\n\n        if self._date_range is not None:\n            start_str, end_str = self._date_range\n            start_date = (\n                datetime.strptime(start_str, DATE_TIME_FORMAT)\n                if start_str.lower() != \"none\"\n                else None\n            )\n            end_date = (\n                datetime.strptime(end_str, DATE_TIME_FORMAT) if end_str.lower() != \"none\" else None\n            )\n            if start_date is not None or end_date is not None:\n                filtered: list[str] = []\n                for file in all_files:\n                    try:","sourceCodeStart":414,"sourceCodeEnd":450,"githubUrl":"https://github.com/xai-org/x-algorithm/blob/24c60942c5c5fdad3a6addffb4c6e6d2f228f04f/phoenix/xrex/data/parquet_recsys.py#L414-L450","documentation":"In index mode, _get_ready_batches_from_index() requires the configured index file to exist on disk; if index_path is None (shouldn't happen after __init__ validation) or os.path.isfile fails, it raises 'Index file {index_path} not found'. This surfaces at data-loading time rather than construction because the file may be created later.","triggerScenarios":"Starting training before the index file was generated; index file on an unmounted NFS volume; typo in index_path; file deleted between config time and read time.","commonSituations":"Race between a job that builds the index and the trainer that consumes it; environment where the data volume mount name changed.","solutions":["Create the index file (one parquet path per line) at the configured path before starting the loader.","Fix mounts/permissions so the path is visible, or correct the path in config.","Retry/wait for the index-producing step if it runs concurrently."],"exampleFix":"# before: files.txt missing\nds = ParquetRecsysDataset(..., index_path='/data/files.txt')\nnext(iter(ds))  # ValueError: not found\n\n# after\nopen('/data/files.txt','w').write('\\n'.join(paths))\nnext(iter(ds))","handlingStrategy":"retry","validationCode":"import os\nif index_path and not os.path.isfile(index_path):\n    build_index_file(topic_dir, index_path)  # write one path per line","typeGuard":null,"tryCatchPattern":"for attempt in range(5):\n    try:\n        batches = ds._get_ready_batches()\n        break\n    except ValueError as e:\n        if 'not found' in str(e) and attempt < 4:\n            time.sleep(30); continue\n        raise","preventionTips":["Generate the index file as the first step of the job, not a manual prerequisite.","Mount/verify the data volume before launch; use absolute paths in config."],"tags":["file-not-found","index-file","dataset"],"backgroundTag":"file-not-found","analyzedSha":"24c60942c5c5fdad3a6addffb4c6e6d2f228f04f","analyzedAt":"2026-08-28T11:40:14.686Z","schemaVersion":2},"datasetVersion":"2026-08-28T16:17:29.566Z"}