apache/beam · error · NotImplementedError

concat(levels)

Error message

concat(levels)

What it means

The Beam DataFrame API's pd.concat wrapper does not support the levels parameter (used with MultiIndex when keys are given). Passing a non-empty levels argument raises NotImplementedError before any processing.

Source

Thrown at sdks/python/apache_beam/dataframe/pandas_top_level_functions.py:102

  @staticmethod
  @frame_base.args_to_kwargs(pd)
  @frame_base.populate_defaults(pd)
  def concat(
      objs,
      axis,
      join,
      ignore_index,
      keys,
      levels,
      names,
      verify_integrity,
      sort,
      copy):

    if ignore_index:
      raise NotImplementedError('concat(ignore_index)')
    if levels:
      raise NotImplementedError('concat(levels)')

    if isinstance(objs, Mapping):
      if keys is None:
        keys = list(objs.keys())
      objs = [objs[k] for k in keys]
    else:
      objs = list(objs)

    if keys is None:
      preserves_partitioning = partitionings.Arbitrary()
    else:
      # Index 0 will be a new index for keys, only partitioning by the original
      # indexes (1 to N) will be preserved.
      nlevels = min(o._expr.proxy().index.nlevels for o in objs)
      preserves_partitioning = partitionings.Index(
          [i for i in range(1, nlevels + 1)])

    deferred_none = expressions.ConstantExpression(None)

View on GitHub (pinned to 12126d8942)

Solutions

1. Omit the levels argument and let the index be built from keys only
2. Construct the desired MultiIndex after concatenation using set_axis or other supported index operations
3. Perform the concat in plain pandas outside the Beam DataFrame transform if levels semantics are required

Example fix

// before
pd.concat([df1, df2], keys=['a','b'], levels=[['a','b','c']])
// after
pd.concat([df1, df2], keys=['a','b'])

if levels:
    levels = None  # not supported by Beam DataFrame concat

try:
    out = pd.concat(objs, keys=keys, levels=levels)
except NotImplementedError:
    out = pd.concat(objs, keys=keys)

When it happens

Trigger: Calling pd.concat(objs, keys=[...], levels=[...]) on deferred Beam dataframes with an explicit levels list.

Common situations: Porting pandas code that builds custom MultiIndex hierarchies via concat with keys+levels.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• concat(ignore_index)
• groupby(as_index=False)
• by
• Assigning an index is not yet supported. Consider using set_
• per-level align

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7992bb75db36b49b. Report an issue: GitHub.