apache/beam · error · NotImplementedError

index.

Error message

index.%s

What it means

_DeferredIndex.__getattr__ raises NotImplementedError('index.<name>') for any attribute accessed on the deferred .index object that is not explicitly implemented (like nlevels). Beam only supports a small whitelist of index operations on deferred frames.

Solutions

1. Avoid index attribute access in deferred (lazy) code; compute needed index info eagerly on a small sample or outside the pipeline.
2. Use supported operations only, or reset_index()/operate on columns instead of the index.
3. Force computation with .to_pandas() on a small test dataset to inspect the index locally.

Example fix

# before
result = beam_df.index.unique()
# after
result = beam_df.reset_index().drop_duplicates(subset='index')['index']

ALLOWED = {'nlevels'}
assert attr in ALLOWED, 'Deferred index attribute %r is unsupported in Beam' % attr

def is_supported_index_attr(name: str) -> bool:
    return name in {'nlevels'}

try:
    val = beam_df.index.<attr>
except NotImplementedError as e:
    if str(e).startswith('index.'):
        val = beam_df.to_pandas().index.<attr>  # only on small/sample data

When it happens

Trigger: Accessing attributes/methods on df.index of a Beam deferred DataFrame such as df.index.name, df.index.unique(), df.index.map(...), or any property not explicitly defined.

Common situations: Pandas code that inspects or manipulates the index after reads; debugging code printing df.index details inside a Beam pipeline; IDE-generated attribute access.

Related errors

• append(ignore_index=True) is order sensitive because it…
• Assigning an index is not yet supported. Consider using…
• by
• concat(ignore_index)
• concat(levels)

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/aed0ef038c9eea99. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/dataframe/frames.py:4902

  @name.setter
  def name(self, value):
    self.names = [value]

  @property
  def ndim(self):
    return self._frame._expr.proxy().index.ndim

  @property
  def dtype(self):
    return self._frame._expr.proxy().index.dtype

  @property
  def nlevels(self):
    return self._frame._expr.proxy().index.nlevels

  def __getattr__(self, name):
    raise NotImplementedError('index.%s' % name)


@populate_not_implemented(pd.core.indexing._LocIndexer)
class _DeferredLoc(object):
  def __init__(self, frame):
    self._frame = frame

  def __getitem__(self, key):
    if isinstance(key, tuple):
      rows, cols = key
      return self[rows][cols]
    elif isinstance(key, list) and key and isinstance(key[0], bool):
      # Aligned by numerical key.
      raise NotImplementedError(type(key))
    elif isinstance(key, list):
      # Select rows, but behaves poorly on missing values.
      raise NotImplementedError(type(key))
    elif isinstance(key, slice):

View on GitHub (pinned to 12126d8942)