apache/beam · error · ValueError

Cannot specify both 'labels' and 'index'/'columns'

Error message

Cannot specify both 'labels' and 'index'/'columns'

What it means

Beam DataFrames' drop() wraps pandas drop with a partitioning-aware implementation. pandas itself forbids passing 'labels' together with 'index'/'columns'; this library re-implements that validation and raises ValueError early so the conflict never reaches the underlying pandas proxy computation.

Source

Thrown at sdks/python/apache_beam/dataframe/frames.py:206

    else:
      return 'indexes=[' + ', '.join(
          '<unnamed>' if ix is None else repr(ix)
          for ix in self.index.names) + ']'

  __array__ = frame_base.wont_implement_method(
      pd.Series, '__array__', reason="non-deferred-result")

  @frame_base.with_docs_from(pd.DataFrame)
  @frame_base.args_to_kwargs(pd.DataFrame)
  @frame_base.populate_defaults(pd.DataFrame)
  @frame_base.maybe_inplace
  def drop(self, labels, axis, index, columns, errors, **kwargs):
    """drop is not parallelizable when dropping from the index and
    ``errors="raise"`` is specified. It requires collecting all data on a single
    node in order to detect if one of the index values is missing."""
    if labels is not None:
      if index is not None or columns is not None:
        raise ValueError("Cannot specify both 'labels' and 'index'/'columns'")
      if axis in (0, 'index'):
        index = labels
        columns = None
      elif axis in (1, 'columns'):
        index = None
        columns = labels
      else:
        raise ValueError(
            "axis must be one of (0, 1, 'index', 'columns'), "
            "got '%s'" % axis)

    if columns is not None:
      # Compute the proxy based on just the columns that are dropped.
      proxy = self._expr.proxy().drop(columns=columns, errors=errors)
    else:
      proxy = self._expr.proxy()

    if index is not None and errors == 'raise':

View on GitHub (pinned to 12126d8942)

Solutions

  1. Pass only one style: either labels= (with axis=) or index=/columns= alone
  2. If dropping column labels, replace labels=...,axis=1 with columns=[...]
  3. If dropping index labels, replace labels=...,axis=0 with index=[...]

Example fix

# before
df.drop(labels='col_a', axis=1, columns=['col_b'])
# after
df.drop(columns=['col_a', 'col_b'])
Defensive patterns

Strategy: validation

Validate before calling

def safe_drop(df, labels=None, axis=None, index=None, columns=None):
    if labels is not None and (index is not None or columns is not None):
        raise ValueError("pass either labels= (with axis) or index=/columns=, not both")
    return df.drop(labels=labels, axis=axis, index=index, columns=columns)

Type guard

def uses_mixed_drop_kwargs(kwargs):
    return 'labels' in kwargs and ('index' in kwargs or 'columns' in kwargs)

Try / catch

try:
    out = df.drop(labels=lbls, axis=axis)
except ValueError as e:
    logging.error("invalid drop() args: %s", e)
    out = df.drop(columns=lbls)  # fallback: assume column labels

Prevention

When it happens

Trigger: Calling df.drop(labels=['a'], columns=1) or df.drop(labels='a', index='x') — i.e. supplying labels and either index or columns simultaneously, regardless of axis.

Common situations: Migrating code from pandas where the mixed call silently fails or is ambiguous; copy-pasted snippets mixing positional and keyword styles; refactor scripts that changed index= to labels= without removing the old keyword.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0a5b870b2a2e4187. Report an issue: GitHub.