microsoft/qlib · error · KeyError

{real_idx} is out of [0, {len(self.idx_map)})

Error message

{real_idx} is out of [0, {len(self.idx_map)})

What it means

`TSDataSampler.__get_idx` maps a flat integer index into an (row, col) position in `idx_map`, whose length equals len(sampler) — the number of valid (datetime, instrument) pairs. An int outside [0, len) raises KeyError.

Source

Thrown at qlib/data/dataset/__init__.py:585

        get the col index and row index of a given sample index in self.idx_df

        Parameters
        ----------
        idx :
            the input of  `__getitem__`

        Returns
        -------
        Tuple[int]:
            the row and col index
        """
        # The the right row number `i` and col number `j` in idx_df
        if isinstance(idx, (int, np.integer)):
            real_idx = idx
            if 0 <= real_idx < len(self.idx_map):
                i, j = self.idx_map[real_idx]  # TODO: The performance of this line is not good
            else:
                raise KeyError(f"{real_idx} is out of [0, {len(self.idx_map)})")
        elif isinstance(idx, tuple):
            # <TSDataSampler object>["datetime", "instruments"]
            date, inst = idx
            date = pd.Timestamp(date)
            i = bisect.bisect_right(self.idx_df.index, date) - 1
            # NOTE: This relies on the idx_df columns sorted in `__init__`
            j = bisect.bisect_left(self.idx_df.columns, inst)
        else:
            raise NotImplementedError(f"This type of input is not supported")
        return i, j

    def __getitem__(self, idx: Union[int, Tuple[object, str], List[int]]):
        """
        # We have two method to get the time-series of a sample
        tsds is a instance of TSDataSampler

        # 1) sample by int index directly
        tsds[len(tsds) - 1]

View on GitHub (pinned to 79633dd950)

Solutions

1. Always derive bounds from the sampler: use `tsds[len(tsds) - 1]` for the last sample.
2. Recompute cached lengths whenever start/end time or instruments change.
3. Check for off-by-one: valid ints are 0..len(tsds)-1.

Example fix

# before
i = 5000  # hard-coded from an older run
sample = tsds[i]

# after
assert 0 <= i < len(tsds)
sample = tsds[i]

def valid_tsds_index(tsds, i: int) -> bool:
    return 0 <= i < len(tsds)

from typing import Union

def is_valid_tsds_index(tsds, idx) -> bool:
    if isinstance(idx, (int,)):
        return 0 <= idx < len(tsds)
    return isinstance(idx, tuple) and len(idx) == 2

When it happens

Trigger: Indexing a TSDataSampler (used by TSDatasetH / sequential models) with a hard-coded int >= number of samples, or reusing an index computed before the dataset shrank (e.g. after `start_time` moved forward), or `tsds[len(tsds)]` off-by-one.

Common situations: Rolling-window training code that caches sample counts; after changing date ranges the count changes and stale indices go out of range; off-by-one in `range(len(tsds)+1)` loops.

Related errors

• Empty data from dataset, please check your dataset config.
• Empty data from dataset, please check your dataset config.
• Empty data from dataset, please check your dataset config.
• This type of input is not supported
• invalid memory_mode `{self.memory_mode}`

AI-assisted analysis of microsoft/qlib@79633dd950 (2026-08-15). Data as JSON: /api/errors/8fa2e26cda34d9eb. Report an issue: GitHub.