apache/beam · error · WontImplementError

get_dummies() of non-categorical type is not supported…

Error message

get_dummies() of non-categorical type is not supported because the type of the output column depends on the data. Please use pd.CategoricalDtype with explicit categories.

What it means

str.get_dummies on a deferred Beam Series raises WontImplementError when the series dtype is not pandas CategoricalDtype. With a non-categorical dtype, the set of dummy columns would be discovered from the data at runtime, making the output columns data-dependent, which Beam's deferred model cannot support.

Solutions

Cast the series to an explicit categorical dtype first: s.astype(pd.CategoricalDtype(categories=[...])).
List all possible categories explicitly so the output columns are known statically.
Build dummy columns manually with boolean expressions per known category (s.str.contains('cat')).
Perform the get_dummies step outside Beam in plain pandas.

Example fix

// before
dummies = s.str.get_dummies(sep=',')

// after
s = s.astype(pd.CategoricalDtype(categories=['a', 'b', 'c']))
dummies = s.str.get_dummies(sep=',')

Defensive patterns

Strategy: validation

Validate before calling

if not isinstance(s._expr.proxy().dtype, pd.CategoricalDtype):
    s = s.astype(pd.CategoricalDtype(categories=known_categories))

Type guard

def is_categorical(s):
    return isinstance(s._expr.proxy().dtype, pd.CategoricalDtype)

Try / catch

try:
    dummies = s.str.get_dummies(sep=',')
except apachebeam.WontImplementError:
    dummies = s.astype(pd.CategoricalDtype(categories=known_categories)).str.get_dummies(sep=',')

Prevention

Cast to CategoricalDtype with an explicit category list before any column-creating string op.
Keep a canonical list of allowed category values in config/code.
Audit string feature-engineering steps for data-dependent output columns.

When it happens

Trigger: Calling s.str.get_dummies(sep=...) where s.dtype is object/string rather than pd.CategoricalDtype on a deferred Beam Series.

Common situations: One-hot encoding free-text tag columns split by a delimiter; porting pandas get_dummies feature engineering to Beam; forgetting to declare the category set up front.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/3fe149930b85bf20. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/dataframe/frames.py:5101

    elif isinstance(repeats, list):
      raise frame_base.WontImplementError(
          "str.repeat(repeats=) repeats must be an int or a DeferredSeries. "
          "Lists are not supported because they make this operation sensitive "
          "to the order of the data.", reason="order-sensitive")
    else:
      raise TypeError("str.repeat(repeats=) value must be an int or a "
                      f"DeferredSeries (encountered {type(repeats)}).")

  @frame_base.with_docs_from(pd.Series.str)
  @frame_base.args_to_kwargs(pd.Series.str)
  def get_dummies(self, **kwargs):
    """
    Series must be categorical dtype. Please cast to ``CategoricalDtype``
    to ensure correct categories.
    """
    dtype = self._expr.proxy().dtype
    if not isinstance(dtype, pd.CategoricalDtype):
      raise frame_base.WontImplementError(
          "get_dummies() of non-categorical type is not supported because "
          "the type of the output column depends on the data. Please use "
          "pd.CategoricalDtype with explicit categories.",
          reason="non-deferred-columns")

    split_cats = [
      cat.split(sep=kwargs.get('sep', '|')) for cat in dtype.categories
    ]

    # Adding the nan category because there could be the case that
    # the data includes NaNs, which is not valid to be casted as a Category,
    # but nevertheless would be broadcasted as a column in get_dummies()
    columns = sorted(set().union(*split_cats))
    if _DUMMY_NAN_COLUMN not in columns:
      columns = columns + [_DUMMY_NAN_COLUMN]

    proxy = pd.DataFrame(columns=columns).astype(int)

View on GitHub (pinned to 12126d8942)