{"record":{"id":"74f1bf46e2a0aee2","repo":"microsoft/qlib","slug":"proc-func-is-not-supported-by-the-storage-type-da","errorCode":null,"errorMessage":"proc_func is not supported by the storage {type(data_storage)}","messagePattern":"proc_func is not supported by the storage (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"qlib/data/dataset/handler.py","lineNumber":315,"sourceCode":"            try:\n                selector = slice(*selector)\n            except ValueError:\n                get_module_logger(\"DataHandlerLP\").info(f\"Fail to converting to query to slice. It will used directly\")\n\n        if isinstance(data_storage, pd.DataFrame):\n            data_df = data_storage\n            if proc_func is not None:\n                # FIXME: fetching by time first will be more friendly to `proc_func`\n                # Copy in case of `proc_func` changing the data inplace....\n                data_df = proc_func(fetch_df_by_index(data_df, selector, level, fetch_orig=self.fetch_orig).copy())\n                data_df = fetch_df_by_col(data_df, col_set)\n            else:\n                # Fetch column  first will be more friendly to SepDataFrame\n                data_df = fetch_df_by_col(data_df, col_set)\n                data_df = fetch_df_by_index(data_df, selector, level, fetch_orig=self.fetch_orig)\n        elif isinstance(data_storage, BaseHandlerStorage):\n            if proc_func is not None:\n                raise ValueError(f\"proc_func is not supported by the storage {type(data_storage)}\")\n            data_df = data_storage.fetch(selector=selector, level=level, col_set=col_set, fetch_orig=self.fetch_orig)\n        else:\n            raise TypeError(f\"data_storage should be pd.DataFrame|HashingStockStorage, not {type(data_storage)}\")\n\n        if squeeze:\n            # squeeze columns\n            data_df = data_df.squeeze()\n            # squeeze index\n            if isinstance(selector, (str, pd.Timestamp)):\n                data_df = data_df.reset_index(level=level, drop=True)\n        return data_df\n\n    def get_cols(self, col_set=DataHandlerABC.CS_ALL) -> list:\n        \"\"\"\n        get the column names\n\n        Parameters\n        ----------","sourceCodeStart":297,"sourceCodeEnd":333,"githubUrl":"https://github.com/microsoft/qlib/blob/79633dd9506ea689e5400dea0197717b5b3d74b7/qlib/data/dataset/handler.py#L297-L333","documentation":"`DataHandler.fetch(..., proc_func=f)` supports a post-fetch callback only when the underlying storage is a plain pd.DataFrame, where it can slice first and copy. When the handler stores data in a `BaseHandlerStorage` (e.g. HashingStockStorage), the fetch is delegated to the storage object which has no proc_func parameter, so passing one raises ValueError.","triggerScenarios":"Constructing/using a DataHandler whose `data_storage` is a HashingStockStorage (hashing-storage mode for large data) and calling `handler.fetch(..., proc_func=my_func)`.","commonSituations":"Workflows that add custom post-processing to fetch() and then switch the handler to hashing storage to save memory; copying example code from DataFrame-storage handlers into hashing-storage configs.","solutions":["Drop proc_func and replicate its logic on the returned DataFrame: `df = handler.fetch(...); df = proc_func(df)`.","Or keep the handler on DataFrame storage (data_storage backed by pd.DataFrame) if proc_func is essential.","Push the transformation into processors configured on the handler instead."],"exampleFix":"# before\ndf = handler.fetch(sel, proc_func=my_func)\n\n# after\ndf = handler.fetch(sel)\ndf = my_func(df)","handlingStrategy":"type-guard","validationCode":"from qlib.data.dataset.handler import BaseHandlerStorage\n\ndef fetch_safe(handler, selector, proc_func=None, **kw):\n    storage = getattr(handler, '_data', None)\n    if proc_func is not None and isinstance(storage, BaseHandlerStorage):\n        return proc_func(handler.fetch(selector, **kw))\n    return handler.fetch(selector, proc_func=proc_func, **kw)","typeGuard":"from qlib.data.dataset.handler import BaseHandlerStorage\n\ndef storage_supports_proc_func(handler) -> bool:\n    return not isinstance(getattr(handler, '_data', None), BaseHandlerStorage)","tryCatchPattern":null,"preventionTips":["Prefer processors over proc_func for reproducible pipelines.","Apply custom transforms to the fetch() result, not inside fetch(), when using hashing storage."],"tags":["data-handler","storage","processors"],"backgroundTag":null,"analyzedSha":"79633dd9506ea689e5400dea0197717b5b3d74b7","analyzedAt":"2026-08-15T07:01:27.511Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}