{"record":{"id":"4a33d6fafdbcfc4e","repo":"QuantConnect/Lean","slug":"unexpected-columns-in-spy-tick-history","errorCode":null,"errorMessage":"Unexpected columns in SPY tick history","messagePattern":"Unexpected columns in SPY tick history","errorType":"exception","errorClass":"AssertionError","httpStatus":null,"severity":"error","filePath":"Algorithm.Python/PandasDataFrameFromMultipleTickTypeTickHistoryRegressionAlgorithm.py","lineNumber":42,"sourceCode":"        self.set_end_date(2013, 10, 8)\n\n        spy = self.add_equity(\"SPY\", Resolution.MINUTE).symbol\n\n        subscriptions = [x for x in self.subscription_manager.subscriptions if x.symbol == spy]\n        if len(subscriptions) != 2:\n            raise AssertionError(f\"Expected 2 subscriptions, but found {len(subscriptions)}\")\n\n        history = pd.DataFrame()\n        try:\n            history = self.history(Tick, spy, timedelta(days=1), Resolution.TICK)\n        except Exception as e:\n            raise AssertionError(f\"History call failed: {e}\")\n\n        if history.shape[0] == 0:\n            raise AssertionError(\"SPY tick history is empty\")\n\n        if not np.array_equal(history.columns.to_numpy(), ['askprice', 'asksize', 'bidprice', 'bidsize', 'exchange', 'lastprice', 'quantity']):\n            raise AssertionError(\"Unexpected columns in SPY tick history\")\n\n        self.quit()\n","sourceCodeStart":24,"sourceCodeEnd":45,"githubUrl":"https://github.com/QuantConnect/Lean/blob/d2c3659f877bfc2b5d9dc0fc89a9c7566f45e892/Algorithm.Python/PandasDataFrameFromMultipleTickTypeTickHistoryRegressionAlgorithm.py#L24-L45","documentation":"Regression assertion that the SPY tick history dataframe has exactly the expected column set: askprice, asksize, bidprice, bidsize, exchange, lastprice, quantity. It uses np.array_equal against the sorted columns, so both the names and the exact set must match. The assertion fires if Lean's PandasData column layout for Tick data changed or columns were added/renamed/missing.","triggerScenarios":"history.columns.to_numpy() does not equal the expected 7-name array. This happens after a change to the Tick property/column mapping in PandasData (e.g. a column renamed, a new field added like 'suspectedfalsy', or a column dropped), or when the returned frame is a different shape (e.g. transposed, or a Series instead of a DataFrame).","commonSituations":"Lean contributors hit this after modifying PandasData.GetFrame / the Tick data column generation, or after a refactor of the BaseData-to-pandas column mapping. Users hit it after upgrading Lean if the tick column contract changed, or when the history call unexpectedly returns a different dtype/layout (e.g. due to a pandas version difference).","solutions":["Log the actual columns (list(history.columns)) to see which name differs.","If a column was renamed/added on purpose in a Lean change, update the expected array to match the new contract.","If the frame shape is wrong (e.g. Series), ensure the request returns a DataFrame — request a single symbol but keep the multi-index path, or inspect dtype.","Align the pandas version with what the dataframe builder expects."],"exampleFix":"# before\nif not np.array_equal(history.columns.to_numpy(), ['askprice','asksize','bidprice','bidsize','exchange','lastprice','quantity']):\n    raise AssertionError(\"Unexpected columns in SPY tick history\")\n\n# after (report the diff so the mismatch is obvious)\nexpected = ['askprice','asksize','bidprice','bidsize','exchange','lastprice','quantity']\nactual = sorted(history.columns.astype(str).tolist())\nif sorted(expected) != actual:\n    raise AssertionError(f\"Unexpected columns in SPY tick history: {actual}\")","handlingStrategy":"validation","validationCode":"# Compare as sorted sets and report the diff\nimport numpy as np\nexpected = ['askprice','asksize','bidprice','bidsize','exchange','lastprice','quantity']\nactual = sorted(map(str, history.columns))\nif sorted(expected) != actual:\n    self.debug(f\"Column mismatch. expected={expected} actual={actual}\")","typeGuard":"def columns_match(history, expected):\n    \"\"\"True when the frame columns equal the expected set (order-insensitive).\"\"\"\n    return set(map(str, history.columns)) == set(expected)","tryCatchPattern":null,"preventionTips":["After a Lean/pandas upgrade, re-derive the expected tick columns from the actual frame.","Compare columns as sets to avoid ordering brittleness.","Log the actual columns when the assertion fires to localize added/renamed fields."],"tags":["quantconnect","lean","history","tick-data","pandas","columns","regression-test","python"],"backgroundTag":null,"analyzedSha":"d2c3659f877bfc2b5d9dc0fc89a9c7566f45e892","analyzedAt":"2026-08-13T13:52:21.013Z","schemaVersion":2},"datasetVersion":"2026-08-13T14:17:21.547Z"}