microsoft/qlib · error · TypeError

data_storage should be pd.DataFrame|HashingStockStorage, not

Error message

data_storage should be pd.DataFrame|HashingStockStorage, not {type(data_storage)}

What it means

DataHandler.init/check of its data storage accepts only two kinds: a pd.DataFrame (classic in-memory handler) or a `BaseHandlerStorage` implementation (e.g. HashingStockStorage). Anything else — a path string, numpy array, dict, None — raises TypeError at fetch time.

Source

Thrown at qlib/data/dataset/handler.py:318

                get_module_logger("DataHandlerLP").info(f"Fail to converting to query to slice. It will used directly")

        if isinstance(data_storage, pd.DataFrame):
            data_df = data_storage
            if proc_func is not None:
                # FIXME: fetching by time first will be more friendly to `proc_func`
                # Copy in case of `proc_func` changing the data inplace....
                data_df = proc_func(fetch_df_by_index(data_df, selector, level, fetch_orig=self.fetch_orig).copy())
                data_df = fetch_df_by_col(data_df, col_set)
            else:
                # Fetch column  first will be more friendly to SepDataFrame
                data_df = fetch_df_by_col(data_df, col_set)
                data_df = fetch_df_by_index(data_df, selector, level, fetch_orig=self.fetch_orig)
        elif isinstance(data_storage, BaseHandlerStorage):
            if proc_func is not None:
                raise ValueError(f"proc_func is not supported by the storage {type(data_storage)}")
            data_df = data_storage.fetch(selector=selector, level=level, col_set=col_set, fetch_orig=self.fetch_orig)
        else:
            raise TypeError(f"data_storage should be pd.DataFrame|HashingStockStorage, not {type(data_storage)}")

        if squeeze:
            # squeeze columns
            data_df = data_df.squeeze()
            # squeeze index
            if isinstance(selector, (str, pd.Timestamp)):
                data_df = data_df.reset_index(level=level, drop=True)
        return data_df

    def get_cols(self, col_set=DataHandlerABC.CS_ALL) -> list:
        """
        get the column names

        Parameters
        ----------
        col_set : str
            select a set of meaningful columns.(e.g. features, columns)

View on GitHub (pinned to 79633dd950)

Solutions

Ensure setup_data actually loads a pd.DataFrame into _data (call super().setup_data in overrides).
Wrap raw arrays: `pd.DataFrame(arr, columns=..., index=pd.MultiIndex.from_product([...]))`.
For out-of-core needs, pass a HashingStockStorage instance.

Example fix

# before
handler._data = np.load('features.npy')

# after
import pandas as pd
handler._data = pd.DataFrame(np.load('features.npy'), columns=['$close'], index=idx)

Defensive patterns

Strategy: type-guard

Validate before calling

import pandas as pd
from qlib.data.dataset.handler import BaseHandlerStorage

def is_valid_storage(obj) -> bool:
    return isinstance(obj, (pd.DataFrame, BaseHandlerStorage))

Type guard

import pandas as pd
from qlib.data.dataset.handler import BaseHandlerStorage

def is_valid_storage(obj) -> bool:
    return isinstance(obj, pd.DataFrame) or isinstance(obj, BaseHandlerStorage)

Prevention

In handler subclasses, always populate _data via super().setup_data.
Validate storage type right after setup_data in custom code.

When it happens

Trigger: Setting `data_handler._data = '/path/to/data'` or constructing a handler subclass whose setup_data leaves _data as a non-DataFrame (e.g. a loader object or None after a failed load).

Common situations: Custom handler subclasses that skip calling super().setup_data, or partial loads that fail silently leaving None; passing file paths where a loaded object is expected.

Related errors

AI-assisted analysis of microsoft/qlib@79633dd950 (2026-08-15). Data as JSON: /api/errors/074e95388bbb89c0. Report an issue: GitHub.