apache/beam · error · NotImplementedError

concat(levels)

Error message

concat(levels)

What it means

The Beam DataFrame API's pd.concat wrapper does not support the levels parameter (used with MultiIndex when keys are given). Passing a non-empty levels argument raises NotImplementedError before any processing.

Solutions

1. Omit the levels argument and let the index be built from keys only
2. Construct the desired MultiIndex after concatenation using set_axis or other supported index operations
3. Perform the concat in plain pandas outside the Beam DataFrame transform if levels semantics are required

Example fix

// before
pd.concat([df1, df2], keys=['a','b'], levels=[['a','b','c']])
// after
pd.concat([df1, df2], keys=['a','b'])

if levels:
    levels = None  # not supported by Beam DataFrame concat

try:
    out = pd.concat(objs, keys=keys, levels=levels)
except NotImplementedError:
    out = pd.concat(objs, keys=keys)

When it happens

Trigger: Calling pd.concat(objs, keys=[...], levels=[...]) on deferred Beam dataframes with an explicit levels list.

Common situations: Porting pandas code that builds custom MultiIndex hierarchies via concat with keys+levels.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

• concat(ignore_index)
• Assigning an index is not yet supported. Consider using…
• by
• corrwith( )
• cross join is not yet implemented…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7992bb75db36b49b. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/dataframe/pandas_top_level_functions.py:102

  @staticmethod
  @frame_base.args_to_kwargs(pd)
  @frame_base.populate_defaults(pd)
  def concat(
      objs,
      axis,
      join,
      ignore_index,
      keys,
      levels,
      names,
      verify_integrity,
      sort,
      copy):

    if ignore_index:
      raise NotImplementedError('concat(ignore_index)')
    if levels:
      raise NotImplementedError('concat(levels)')

    if isinstance(objs, Mapping):
      if keys is None:
        keys = list(objs.keys())
      objs = [objs[k] for k in keys]
    else:
      objs = list(objs)

    if keys is None:
      preserves_partitioning = partitionings.Arbitrary()
    else:
      # Index 0 will be a new index for keys, only partitioning by the original
      # indexes (1 to N) will be preserved.
      nlevels = min(o._expr.proxy().index.nlevels for o in objs)
      preserves_partitioning = partitionings.Index(
          [i for i in range(1, nlevels + 1)])

    deferred_none = expressions.ConstantExpression(None)

View on GitHub (pinned to 12126d8942)