apache/beam · error · NotImplementedError

Assigning an index is not yet supported. Consider using set_

Error message

Assigning an index is not yet supported. Consider using set_index() instead.

What it means

Assigning to the .index attribute of a Beam DeferredFrame (df.index = ...) is an order-sensitive operation the distributed implementation cannot support, so the index setter raises NotImplementedError and points to set_index().

Source

Thrown at sdks/python/apache_beam/dataframe/frames.py:1201

          "on the full dataset."),
      preserves_partition_by=expressions.partitionings.Singleton())

  @property  # type: ignore
  @frame_base.with_docs_from(pd.DataFrame)
  def ndim(self):
    return self._expr.proxy().ndim

  @property  # type: ignore
  @frame_base.with_docs_from(pd.DataFrame)
  def index(self):
    return _DeferredIndex(self)

  @index.setter
  def _set_index(self, value):
    # TODO: assigning the index is generally order-sensitive, but we could
    # support it in some rare cases, e.g. when assigning the index from one
    # of a DataFrame's columns
    raise NotImplementedError(
        "Assigning an index is not yet supported. "
        "Consider using set_index() instead.")

  reindex = frame_base.wont_implement_method(
      pd.DataFrame, 'reindex', reason="order-sensitive")

  hist = frame_base.wont_implement_method(
      pd.DataFrame, 'hist', reason="plotting-tools")

  attrs = property(
      fget=frame_base.wont_implement_method(
          pd.DataFrame, 'attrs', reason='experimental'),
      fset=frame_base.wont_implement_method(
          pd.DataFrame, 'attrs', reason='experimental'),
  )

  reorder_levels = frame_base._proxy_method(
      'reorder_levels',

View on GitHub (pinned to 12126d8942)

Solutions

Use df.set_index('col') to build the index from a column
Use reset_index(drop=True) to replace the index instead of assigning
Convert to pandas (df.to_pandas()) only if the data fits in memory, then assign

Example fix

// before
df.index = df['id']
// after
df = df.set_index('id')

Defensive patterns

Strategy: fallback

Validate before calling

# avoid the pattern entirely
assert not isinstance(value, pd.Index) or True  # never do: df.index = value

Try / catch

try:
    df.index = new_index
except NotImplementedError:
    df = df.set_index(new_index.name) if hasattr(new_index, 'name') else df.reset_index(drop=True)

Prevention

Never assign to .index on DeferredFrames
Use set_index()/reset_index() exclusively
Grep for '.index =' when porting pandas code to Beam

When it happens

Trigger: Writing df.index = new_index or df.index = df['col'] on a Beam DataFrame; assigning index inside a pipeline transformation.

Common situations: Direct pandas ports that reassign the index; attempts to overwrite the index with a column; migrations where pandas code did df.index = pd.RangeIndex(...).

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d4d7899ec57129cf. Report an issue: GitHub.