{"record":{"id":"8fa2e26cda34d9eb","repo":"microsoft/qlib","slug":"real-idx-is-out-of-0-len-self-idx-map","errorCode":null,"errorMessage":"{real_idx} is out of [0, {len(self.idx_map)})","messagePattern":"(.+?) is out of \\[0, (.+?)\\)","errorType":"exception","errorClass":"KeyError","httpStatus":null,"severity":"error","filePath":"qlib/data/dataset/__init__.py","lineNumber":585,"sourceCode":"        get the col index and row index of a given sample index in self.idx_df\n\n        Parameters\n        ----------\n        idx :\n            the input of  `__getitem__`\n\n        Returns\n        -------\n        Tuple[int]:\n            the row and col index\n        \"\"\"\n        # The the right row number `i` and col number `j` in idx_df\n        if isinstance(idx, (int, np.integer)):\n            real_idx = idx\n            if 0 <= real_idx < len(self.idx_map):\n                i, j = self.idx_map[real_idx]  # TODO: The performance of this line is not good\n            else:\n                raise KeyError(f\"{real_idx} is out of [0, {len(self.idx_map)})\")\n        elif isinstance(idx, tuple):\n            # <TSDataSampler object>[\"datetime\", \"instruments\"]\n            date, inst = idx\n            date = pd.Timestamp(date)\n            i = bisect.bisect_right(self.idx_df.index, date) - 1\n            # NOTE: This relies on the idx_df columns sorted in `__init__`\n            j = bisect.bisect_left(self.idx_df.columns, inst)\n        else:\n            raise NotImplementedError(f\"This type of input is not supported\")\n        return i, j\n\n    def __getitem__(self, idx: Union[int, Tuple[object, str], List[int]]):\n        \"\"\"\n        # We have two method to get the time-series of a sample\n        tsds is a instance of TSDataSampler\n\n        # 1) sample by int index directly\n        tsds[len(tsds) - 1]","sourceCodeStart":567,"sourceCodeEnd":603,"githubUrl":"https://github.com/microsoft/qlib/blob/79633dd9506ea689e5400dea0197717b5b3d74b7/qlib/data/dataset/__init__.py#L567-L603","documentation":"`TSDataSampler.__get_idx` maps a flat integer index into an (row, col) position in `idx_map`, whose length equals len(sampler) — the number of valid (datetime, instrument) pairs. An int outside [0, len) raises KeyError.","triggerScenarios":"Indexing a TSDataSampler (used by TSDatasetH / sequential models) with a hard-coded int >= number of samples, or reusing an index computed before the dataset shrank (e.g. after `start_time` moved forward), or `tsds[len(tsds)]` off-by-one.","commonSituations":"Rolling-window training code that caches sample counts; after changing date ranges the count changes and stale indices go out of range; off-by-one in `range(len(tsds)+1)` loops.","solutions":["Always derive bounds from the sampler: use `tsds[len(tsds) - 1]` for the last sample.","Recompute cached lengths whenever start/end time or instruments change.","Check for off-by-one: valid ints are 0..len(tsds)-1."],"exampleFix":"# before\ni = 5000  # hard-coded from an older run\nsample = tsds[i]\n\n# after\nassert 0 <= i < len(tsds)\nsample = tsds[i]","handlingStrategy":"validation","validationCode":"def valid_tsds_index(tsds, i: int) -> bool:\n    return 0 <= i < len(tsds)","typeGuard":"from typing import Union\n\ndef is_valid_tsds_index(tsds, idx) -> bool:\n    if isinstance(idx, (int,)):\n        return 0 <= idx < len(tsds)\n    return isinstance(idx, tuple) and len(idx) == 2","tryCatchPattern":null,"preventionTips":["Never hard-code sample indices; derive them from len(tsds).","Recompute index bounds after any change to instruments or time range."],"tags":["dataset","indexing","time-series"],"backgroundTag":null,"analyzedSha":"79633dd9506ea689e5400dea0197717b5b3d74b7","analyzedAt":"2026-08-15T07:01:27.511Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}