{"record":{"id":"6a23ec75782e6935","repo":"QuantConnect/Lean","slug":"could-not-unstack-df-columns-join-df-colum","errorCode":null,"errorMessage":"Could not unstack df. Columns: {', '.join(df.columns)} | {column}","messagePattern":"Could not unstack df\\. Columns: (.+?) \\| (.+?)","errorType":"exception","errorClass":"AssertionError","httpStatus":null,"severity":"error","filePath":"Algorithm.Python/PandasDataFrameHistoryAlgorithm.py","lineNumber":99,"sourceCode":"        # we can use these TradeBars to initialize indicators or perform other math\n        self.spy_daily_sma.reset()\n        for index, trade_bar in trade_bar_history.loc[\"SPY\"].iterrows():\n            self.spy_daily_sma.update(index, trade_bar[\"close\"])\n\n        # we can loop over the return values from these functions and we'll get Quandl data\n        # this can be used in much the same way as the trade_bar_history above\n        self.spy_daily_sma.reset()\n        for index, quandl in quandl_history.loc[\"CHRIS/CME_SP1\"].iterrows():\n            self.spy_daily_sma.update(index, quandl[\"settle\"])\n\n        self.set_holdings(self.eur, 1)\n\n    def assert_history_index(self, df, column, expected, ticker, symbol):\n\n        if df.empty:\n            raise AssertionError(f\"Empty history data frame for {symbol}\")\n        if column not in df:\n            raise AssertionError(f\"Could not unstack df. Columns: {', '.join(df.columns)} | {column}\")\n\n        value = df.iat[0,0]\n        df2 = df.xs(df.index.get_level_values('time')[0], level='time')\n        df3 = df[column].unstack(level=0)\n\n        try:\n\n            # str(Symbol.ID)\n            self.assert_history_count(f\"df.iloc[0]\", df.iloc[0], len(df.columns))\n            self.assert_history_count(f\"df.loc[str({symbol.id})]\", df.loc[str(symbol.id)], expected)\n            self.assert_history_count(f\"df.xs(str({symbol.id}))\", df.xs(str(symbol.id)), expected)\n            self.assert_history_count(f\"df.at[(str({symbol.id}),), '{column}']\", list(df.at[(str(symbol.id),), column]), expected)\n            self.assert_history_count(f\"df2.loc[str({symbol.id})]\", df2.loc[str(symbol.id)], len(df2.columns))\n            self.assert_history_count(f\"df3[str({symbol.id})]\", df3[str(symbol.id)], expected)\n            self.assert_history_count(f\"df3.get(str({symbol.id}))\", df3.get(str(symbol.id)), expected)\n\n            # str(Symbol)\n            self.assert_history_count(f\"df.loc[str({symbol})]\", df.loc[str(symbol)], expected)","sourceCodeStart":81,"sourceCodeEnd":117,"githubUrl":"https://github.com/QuantConnect/Lean/blob/d2c3659f877bfc2b5d9dc0fc89a9c7566f45e892/Algorithm.Python/PandasDataFrameHistoryAlgorithm.py#L81-L117","documentation":"Guard in assert_history_index that the expected column exists in the dataframe before unstacking/accessing it. The helper later does df[column].unstack(level=0) and df.at[(...), column], so the column must be present. The assertion fires when the history frame's columns do not include the requested column name, indicating the unstack/indexing strategy will not work for this data.","triggerScenarios":"column not in df.columns for a history dataframe. This happens when the data type's properties do not include the expected column (e.g. asking for 'settle' on data that does not expose it), when the dataframe is multi-indexed and the column lives at a different level, or when Lean's PandasData column generation for that type changed/renamed the field.","commonSituations":"A custom data type (e.g. QuandlFuture expecting 'Settle') no longer exposes the column after a rename or a change to value_column_name. The history frame came back with a different column layout after a Lean or pandas upgrade. The caller passed the wrong column name for the requested data type.","solutions":["Log df.columns to see the actual available columns and pick the correct name.","If using custom data, verify the data type's properties and value_column_name still produce the expected column.","After a Lean/pandas upgrade, re-check the column-generation contract for that data type and update the expected column name.","Confirm the dataframe is at the expected index level (symbol/time) so column lookup is valid."],"exampleFix":"# before\nif column not in df:\n    raise AssertionError(f\"Could not unstack df. Columns: {', '.join(df.columns)} | {column}\")\n\n# after (report columns in a stable, sorted form and hint at the mismatch)\nif column not in df.columns:\n    raise AssertionError(\n        f\"Could not unstack df; '{column}' missing. \"\n        f\"Available columns: {', '.join(map(str, sorted(df.columns)))}\")","handlingStrategy":"validation","validationCode":"# Verify the column exists (and report actual columns) before unstacking\ndef assert_history_index(self, df, column, expected, ticker, symbol):\n    if column not in df.columns:\n        self.debug(f\"'{column}' not in {sorted(map(str, df.columns))} for {symbol}\")\n        return","typeGuard":"def has_column(df, column):\n    \"\"\"True when the dataframe exposes the requested column.\"\"\"\n    return df is not None and column in df.columns","tryCatchPattern":null,"preventionTips":["Log df.columns to confirm available field names before accessing.","Keep custom-data value_column_name consistent with the asserted column.","Re-check column contracts after a Lean/pandas upgrade."],"tags":["quantconnect","lean","history","pandas","columns","custom-data","python"],"backgroundTag":null,"analyzedSha":"d2c3659f877bfc2b5d9dc0fc89a9c7566f45e892","analyzedAt":"2026-08-13T13:52:21.013Z","schemaVersion":2},"datasetVersion":"2026-08-13T14:17:21.547Z"}