{"record":{"id":"9b77b9e87478c293","repo":"apache/beam","slug":"method-name-of-non-categorical-type-is-not-supported-because","errorCode":null,"errorMessage":"{method_name}() of non-categorical type is not supported because the type of the output column depends on the data. Please use pd.CategoricalDtype with explicit categories.","messagePattern":"(.+?)\\(\\) of non-categorical type is not supported because the type of the output column depends on the data\\. Please use pd\\.CategoricalDtype with explicit categories\\.","errorType":"exception","errorClass":"WontImplementError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/dataframe/frames.py","lineNumber":5146,"sourceCode":"            requires_partition_by=partitionings.Arbitrary(),\n            preserves_partition_by=partitionings.Arbitrary()))\n\n  def _split_helper(self, rsplit=False, **kwargs):\n    expand = kwargs.get('expand', False)\n\n    if not expand:\n      # Not creating separate columns\n      proxy = self._expr.proxy()\n      if not rsplit:\n        func = lambda s: pd.concat([proxy, s.str.split(**kwargs)])\n      else:\n        func = lambda s: pd.concat([proxy, s.str.rsplit(**kwargs)])\n    else:\n      # Creating separate columns, so must be more strict on dtype\n      dtype = self._expr.proxy().dtype\n      if not isinstance(dtype, pd.CategoricalDtype):\n        method_name = 'rsplit' if rsplit else 'split'\n        raise frame_base.WontImplementError(\n            f\"{method_name}() of non-categorical type is not supported because \"\n            \"the type of the output column depends on the data. Please use \"\n            \"pd.CategoricalDtype with explicit categories.\",\n            reason=\"non-deferred-columns\")\n\n      # Split the categories\n      split_cats = dtype.categories.str.split(**kwargs)\n\n      # Count the number of new columns to create for proxy\n      max_splits = len(max(split_cats, key=len))\n      proxy = pd.DataFrame(columns=range(max_splits))\n\n      def func(s):\n        if not rsplit:\n          result = s.str.split(**kwargs)\n        else:\n          result = s.str.rsplit(**kwargs)\n        result[~result.isna()].replace(np.nan, value=None)","sourceCodeStart":5128,"sourceCodeEnd":5164,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/dataframe/frames.py#L5128-L5164","documentation":"str.split/str.rsplit with expand=True (separate output columns) raises WontImplementError when the series dtype is not CategoricalDtype. When splitting into distinct columns, the column names and count depend on the categories found in the data; Beam requires a categorical dtype so the resulting schema is static.","triggerScenarios":"s.str.split(sep, expand=True) or s.str.rsplit(sep, expand=True) where s.dtype is not pd.CategoricalDtype on a deferred Beam Series (the error text names 'split' or 'rsplit' via method_name).","commonSituations":"Splitting a delimited field into multiple DataFrame columns in a Beam pipeline; porting pandas expand=True splits; forgetting that Beam requires declared categories for column-creating operations.","solutions":["Cast to categorical first: s.astype(pd.CategoricalDtype(categories=[...])) before calling split/rsplit with expand=True.","Use expand=False (default) to get a list-like single column, then extract parts with .str[i] into named columns.","Create each output column explicitly with .str.split(...).str.get(0), .str.get(1), etc.","Do the expand split in plain pandas outside the pipeline."],"exampleFix":"// before\nparts = s.str.split('-', expand=True)\n\n// after\nparts0 = s.str.split('-').str.get(0)\nparts1 = s.str.split('-').str.get(1)","handlingStrategy":"validation","validationCode":"if expand and not isinstance(s._expr.proxy().dtype, pd.CategoricalDtype):\n    raise ValueError(\"cast to CategoricalDtype before split(expand=True)\")","typeGuard":"def is_categorical(s):\n    return isinstance(s._expr.proxy().dtype, pd.CategoricalDtype)","tryCatchPattern":"try:\n    parts = s.str.split(sep, expand=True)\nexcept apachebeam.WontImplementError:\n    parts0 = s.str.split(sep).str.get(0)\n    parts1 = s.str.split(sep).str.get(1)","preventionTips":["Prefer expand=False plus .str.get(i) column extraction in Beam.","Declare categories via astype(pd.CategoricalDtype(...)) when expand=True is needed.","Check the proxy dtype with s.dtype at pipeline-construction time."],"tags":["apache-beam","dataframe","deferred-columns","categorical","pandas"],"backgroundTag":"unsupported-operation","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}