{"record":{"id":"6ed6bdc062580d7b","repo":"apache/beam","slug":"unstack-is-only-supported-on-dataframes-if-unstacked-level","errorCode":null,"errorMessage":"unstack() is only supported on DataFrames if unstacked level is a categorical or boolean column","messagePattern":"unstack\\(\\) is only supported on DataFrames if unstacked level is a categorical or boolean column","errorType":"exception","errorClass":"WontImplementError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/dataframe/frames.py","lineNumber":1005,"sourceCode":"            \"unstack() is not supported when using pandas < 1.2.0\\n\"\n            \"Please upgrade to pandas 1.2.0 or higher to use this operation.\")\n      return frame_base.DeferredFrame.wrap(\n          expressions.ComputedExpression(\n              'unstack', lambda s: s.unstack(**kwargs), [self._expr],\n              requires_partition_by=partitionings.Index()))\n    else:\n      # Unstacking MultiIndex objects\n      idx = self._expr.proxy().index\n\n      # Converting level (int, str, or combination) to a list of number levels\n      level_list = level if isinstance(level, list) else [level]\n      level_number_list = [idx._get_level_number(l) for l in level_list]\n\n      # Checking if levels provided are of CategoricalDtype\n      if not all(isinstance(idx.levels[l].dtype, (pd.CategoricalDtype,\n                                                  pd.BooleanDtype))\n                 for l in level_number_list):\n        raise frame_base.WontImplementError(\n            \"unstack() is only supported on DataFrames if unstacked level \"\n            \"is a categorical or boolean column\",\n            reason=\"non-deferred-columns\")\n      else:\n        tmp = self._expr.proxy().unstack(**kwargs)\n        if isinstance(tmp.columns, pd.MultiIndex):\n          levels = []\n          for i in range(tmp.columns.nlevels):\n            level = tmp.columns.levels[i]\n            levels.append(level)\n          col_idx = pd.MultiIndex.from_product(levels)\n        else:\n          if tmp.columns.dtype == 'boolean':\n            col_idx = pd.Index([False, True], dtype='boolean')\n          else:\n            col_idx = pd.CategoricalIndex(tmp.columns.categories)\n\n        if isinstance(self._expr.proxy(), pd.Series):","sourceCodeStart":987,"sourceCodeEnd":1023,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/dataframe/frames.py#L987-L1023","documentation":"For a DeferredDataFrame with a MultiIndex, unstack is only supported when every level being unstacked has CategoricalDtype or BooleanDtype. A non-categorical/boolean level would create result columns whose existence and order depend on the data, which Beam's deferred model cannot represent (reason \"non-deferred-columns\").","triggerScenarios":"Calling unstack() or unstack(level=...) on a DeferredDataFrame with a MultiIndex where at least one of the requested levels is a plain (object/int) dtype rather than categorical or boolean.","commonSituations":"Unstacking string or integer index levels migrated from pandas; forgetting to convert the level to a categorical dtype before the pipeline.","solutions":["Cast the level to categorical before unstacking: df.index = df.index.set_levels(df.index.levels[l].astype('category'), level=l).","Cast boolean-like levels to pandas BooleanDtype ('boolean').","If the dtype can't be categorical, do the unstack after to_pandas()."],"exampleFix":"// before\ndf.unstack(level='city')  # 'city' is object dtype -> WontImplementError\n// after\ndf.index = df.index.set_levels(df.index.levels[df.index.names.index('city')].astype('category'), level='city')\ndf.unstack(level='city')","handlingStrategy":"validation","validationCode":"import pandas as pd\nlevel_numbers = [idx._get_level_number(l) for l in level_list]\nif not all(isinstance(idx.levels[l].dtype, (pd.CategoricalDtype, pd.BooleanDtype)) for l in level_numbers):\n    raise ValueError(\"unstack levels must be categorical or boolean dtype in Beam\")","typeGuard":"def is_unstackable_level(level_dtype):\n    return isinstance(level_dtype, (pd.CategoricalDtype, pd.BooleanDtype))","tryCatchPattern":"from apache_beam.dataframe import frame_base\ntry:\n    out = df.unstack(level=levels)\nexcept frame_base.WontImplementError:\n    out = df.to_pandas().unstack(level=levels)","preventionTips":["Cast MultiIndex levels to 'category' (or 'boolean') dtype before unstacking.","Inspect index.dtypes of the deferred frame's proxy before planning an unstack.","Prefer joins/pivots on categorical keys when designing the pipeline."],"tags":["apache-beam","dataframe","pandas","dtype"],"backgroundTag":"unsupported-dtype","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}