{"record":{"id":"d0a0d79895c5b689","repo":"microsoft/qlib","slug":"unrecognized-data-shape-shape","errorCode":null,"errorMessage":"Unrecognized data shape: {shape}","messagePattern":"Unrecognized data shape: (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"qlib/rl/data/pickle_styled.py","lineNumber":67,"sourceCode":"            \"$close\",\n            \"$vwap\",\n            \"$bid\",\n            \"$ask\",\n            \"$volume\",\n            \"$bidV\",\n            \"$bidV1\",\n            \"$bidV3\",\n            \"$bidV5\",\n            \"$askV\",\n            \"$askV1\",\n            \"$askV3\",\n            \"$askV5\",\n        ]\n    if shape == 6:\n        return [\"$high\", \"$low\", \"$open\", \"$close\", \"$vwap\", \"$volume\"]\n    elif shape == 5:\n        return [\"$high\", \"$low\", \"$open\", \"$close\", \"$volume\"]\n    raise ValueError(f\"Unrecognized data shape: {shape}\")\n\n\ndef _find_pickle(filename_without_suffix: Path) -> Path:\n    suffix_list = [\".pkl\", \".pkl.backtest\"]\n    paths: List[Path] = []\n    for suffix in suffix_list:\n        path = filename_without_suffix.parent / (filename_without_suffix.name + suffix)\n        if path.exists():\n            paths.append(path)\n    if not paths:\n        raise FileNotFoundError(f\"No file starting with '{filename_without_suffix}' found\")\n    if len(paths) > 1:\n        raise ValueError(f\"Multiple paths are found with prefix '{filename_without_suffix}': {paths}\")\n    return paths[0]\n\n\n@lru_cache(maxsize=10)  # 10 * 40M = 400MB\ndef _read_pickle(filename_without_suffix: Path) -> pd.DataFrame:","sourceCodeStart":49,"sourceCodeEnd":85,"githubUrl":"https://github.com/microsoft/qlib/blob/79633dd9506ea689e5400dea0197717b5b3d74b7/qlib/rl/data/pickle_styled.py#L49-L85","documentation":"In qlib/rl/data/pickle_styled.py:67, when process data is loaded from pickle files the column names are inferred from the array's feature dimension. Only shape 6 (high/low/open/close/vwap/volume), shape 5 (high/low/open/close/volume), and an 11/12-column order-book case (bid/ask levels) are recognized. Any other last-dimension size raises ValueError('Unrecognized data shape: <shape>').","triggerScenarios":"Loading a pickle-styled process data file whose feature axis is not 5, 6, or the recognized book depth; hand-crafted pickles with extra columns or a different column order/count; data generated by a modified/different-versioned preprocessing script.","commonSituations":"See trigger scenarios.","solutions":["Regenerate the pickle data with an accepted schema: 6 features ($high,$low,$open,$close,$vwap,$volume), 5 features (no vwap), or the standard order-book layout","If you need custom columns, construct the DataFrame with explicit column names yourself instead of relying on shape inference","Check for accidental extra columns/rows introduced by your preprocessing"],"exampleFix":"# before\n# pickle with 7 columns -> ValueError('Unrecognized data shape: (7,)')\n\n# after\n# keep the standard 6 columns when dumping\ndf = df[['$high', '$low', '$open', '$close', '$vwap', '$volume']]\ndf.to_pickle('data.pkl')","handlingStrategy":"validation","validationCode":"shape = arr.shape[-1]\nassert shape in (5, 6) or shape >= 11, f'data feature dimension {shape} not recognized'","typeGuard":null,"tryCatchPattern":"try:\n    data = pickle_styled.load(path)\nexcept ValueError as e:\n    if 'Unrecognized data shape' in str(e):\n        raise ValueError('regenerate pickle with standard 5/6/book columns') from e\n    raise","preventionTips":["Pin the preprocessing script version that produced RL pickle data","Validate dumped arrays' feature dimension immediately after generation"],"tags":["qlib","rl","data","pickle","schema","value-error"],"backgroundTag":null,"analyzedSha":"79633dd9506ea689e5400dea0197717b5b3d74b7","analyzedAt":"2026-08-15T07:01:27.511Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}