apache/beam · error · WontImplementError

append() only accepts DeferredDataFrame instances, received

Error message

append() only accepts DeferredDataFrame instances, received {type(other)}

What it means

DeferredDataFrame.append only accepts another DeferredDataFrame as `other`; passing any other object (plain pd.DataFrame, pd.Series, dict, list) raises a WontImplementError. Beam cannot align eager pandas objects with distributed deferred data without order-sensitive operations.

Solutions

Wrap the eager pandas object first: other = apache_beam.dataframe.frames.DeferredFrame.wrap(other) or convert it to a DeferredDataFrame.
Combine the data upstream (in plain pandas) before creating the deferred frame.
Perform the append inside a Apply/Map stage where the data is eager.

Example fix

// before
ddf.append(pd_df)

// after
from apache_beam.dataframe.frame_base import DeferredFrame
ddf.append(DeferredFrame.wrap(pd_df))

Defensive patterns

Strategy: type-guard

Validate before calling

from apache_beam.dataframe.frame_base import DeferredFrame
if not isinstance(other, DeferredFrame):
    other = DeferredFrame.wrap(other)

Type guard

def is_deferred_frame(x) -> bool:
    from apache_beam.dataframe.frame_base import DeferredFrame
    return isinstance(x, DeferredFrame)

Try / catch

from apache_beam.dataframe import frame_base
try:
    ddf = ddf.append(other)
except frame_base.WontImplementError:
    ddf = ddf.append(DeferredFrame.wrap(other))

Prevention

Wrap every eager pandas object with DeferredFrame.wrap before mixing with deferred frames.
Keep eager and distributed data separate in the pipeline design.
Add assertions on argument types in helper functions.

When it happens

Trigger: Calling `ddf.append(pd.DataFrame(...))` or `ddf.append(some_series_or_dict)` where `other` is not a DeferredDataFrame instance.

Common situations: Mixing a plain pandas DataFrame loaded in driver memory with a Beam deferred DataFrame, e.g. appending a small lookup table to a streaming/distributed frame.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7484a571bd5431c6. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/dataframe/frames.py:2632

            [self._expr, other._expr],
            requires_partition_by=requires_partition_by,
            preserves_partition_by=partitionings.Arbitrary()))

  @frame_base.with_docs_from(pd.DataFrame, removed_method=PD_VERSION >= (2, 0))
  @frame_base.args_to_kwargs(pd.DataFrame, removed_method=PD_VERSION >= (2, 0))
  @frame_base.populate_defaults(pd.DataFrame,
                                removed_method=PD_VERSION >= (2, 0))
  def append(self, other, ignore_index, verify_integrity, sort, **kwargs):
    """``ignore_index=True`` is not supported, because it requires generating an
    order-sensitive index."""
    if PD_VERSION >= (2, 0):
      raise frame_base.WontImplementError('append() was removed in Pandas 2.0.')
    if not isinstance(other, DeferredDataFrame):
      raise frame_base.WontImplementError(
          "append() only accepts DeferredDataFrame instances, received " +
          str(type(other)))
    if ignore_index:
      raise frame_base.WontImplementError(
          "append(ignore_index=True) is order sensitive because it requires "
          "generating a new index based on the order of the data.",
          reason="order-sensitive")

    if verify_integrity:
      # We can verify the index is non-unique within index partitioned data.
      requires = partitionings.Index()
    else:
      requires = partitionings.Arbitrary()

    return frame_base.DeferredFrame.wrap(
        expressions.ComputedExpression(
            'append',
            lambda s, other: s.append(other, sort=sort,
                                      verify_integrity=verify_integrity,
                                      **kwargs),
            [self._expr, other._expr],
            requires_partition_by=requires,

View on GitHub (pinned to 12126d8942)