{"record":{"id":"39d135d125a90dc4","repo":"microsoft/qlib","slug":"i-start-index-is-storage-start-index","errorCode":null,"errorMessage":"{i}: start index is {storage_start_index}","messagePattern":"(.+?): start index is (.+?)","errorType":"exception","errorClass":"IndexError","httpStatus":null,"severity":"error","filePath":"qlib/data/storage/file_storage.py","lineNumber":360,"sourceCode":"            return None\n        # The next  data appending index point will be  `end_index + 1`\n        return self.start_index + len(self) - 1\n\n    def __getitem__(self, i: Union[int, slice]) -> Union[Tuple[int, float], pd.Series]:\n        if not self.uri.exists():\n            if isinstance(i, int):\n                return None, None\n            elif isinstance(i, slice):\n                return pd.Series(dtype=np.float32)\n            else:\n                raise TypeError(f\"type(i) = {type(i)}\")\n\n        storage_start_index = self.start_index\n        storage_end_index = self.end_index\n        with self.uri.open(\"rb\") as fp:\n            if isinstance(i, int):\n                if storage_start_index > i:\n                    raise IndexError(f\"{i}: start index is {storage_start_index}\")\n                fp.seek(4 * (i - storage_start_index) + 4)\n                return i, struct.unpack(\"f\", fp.read(4))[0]\n            elif isinstance(i, slice):\n                start_index = storage_start_index if i.start is None else i.start\n                end_index = storage_end_index if i.stop is None else i.stop - 1\n                si = max(start_index, storage_start_index)\n                if si > end_index:\n                    return pd.Series(dtype=np.float32)\n                fp.seek(4 * (si - storage_start_index) + 4)\n                # read n bytes\n                count = end_index - si + 1\n                data = np.frombuffer(fp.read(4 * count), dtype=\"<f\")\n                return pd.Series(data, index=pd.RangeIndex(si, si + len(data)))\n            else:\n                raise TypeError(f\"type(i) = {type(i)}\")\n\n    def __len__(self) -> int:\n        self.check()","sourceCodeStart":342,"sourceCodeEnd":378,"githubUrl":"https://github.com/microsoft/qlib/blob/79633dd9506ea689e5400dea0197717b5b3d74b7/qlib/data/storage/file_storage.py#L342-L378","documentation":"Raised by FileFeatureStorage.__getitem__ (qlib/data/storage/file_storage.py) when an integer index i is smaller than the storage's start_index. Bin files only cover rows [start_index, end_index]; querying an index before the first stored row cannot be answered, so an IndexError names the requested index and the actual start index. Slices, by contrast, are clamped to the available range and never raise.","triggerScenarios":"feature_storage[1234] when that instrument's bin file starts at start_index=1500 (e.g. the stock listed later than the requested calendar position). Common when computing calendar-relative indices for an instrument whose data begins mid-calendar.","commonSituations":"Hand-computing absolute calendar indices and applying them to every instrument equally; early-listed vs late-listed instruments in a cross-section; code that assumes all bin files start at the same index.","solutions":["Query per-instrument ranges: use storage.start_index / storage.end_index to clamp, or use slice-based access which clamps automatically.","Prefer qlib's expression/data APIs (D.features, Cal.calendar) over raw storage indexing; they handle per-instrument coverage.","If the data should exist that far back, re-dump the instrument's bin file with a longer history."],"exampleFix":"# before\nfirst = feature_storage[calendar_index]  # calendar_index < storage.start_index -> IndexError\n\n# after\nstart = feature_storage.start_index\nfirst = feature_storage[max(calendar_index, start)]\n# or simply use a slice, which clamps:\nfirst = feature_storage[calendar_index:calendar_index + 1]","handlingStrategy":"validation","validationCode":"i = max(int(i), storage.start_index)\nif i > storage.end_index:\n    i = storage.start_index  # or handle 'no data' explicitly","typeGuard":"def index_within_storage(i: int, storage) -> bool:\n    return storage.start_index <= i <= storage.end_index","tryCatchPattern":"try:\n    _, val = storage[i]\nexcept IndexError as e:\n    if \"start index is\" in str(e):\n        _, val = storage[storage.start_index]  # clamp to first available row\n    else:\n        raise","preventionTips":["Always clamp indices against storage.start_index/end_index before int access.","Prefer slice access, which clamps automatically, or qlib's D.features API."],"tags":["qlib","storage","indexing","out-of-range"],"backgroundTag":null,"analyzedSha":"79633dd9506ea689e5400dea0197717b5b3d74b7","analyzedAt":"2026-08-15T07:01:27.511Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}