apache/beam · error · WontImplementError

not dataframes or series

Error message

not dataframes or series

What it means

apache_beam.dataframe.io.to_json only supports deferring a pd.DataFrame or pd.Series to JSON via the Beam pipeline. When the orient argument is not supplied, it inspects the object's proxy type; if the expression wraps neither a DataFrame nor a Series, it raises WontImplementError because there is no defined 'orient' for such an object and no deferred implementation exists.

Solutions

1. Pass an explicit orient='columns' (DataFrame) or orient='index' (Series) argument to to_json.
2. Ensure the argument is a real deferred DataFrame or Series from beam.dataframe.io read_* calls.
3. Convert via .to_frame() / .to_series() so the proxy type is a pd.DataFrame or pd.Series.
4. If the object genuinely is neither, write it out with a plain Beam PTransform (e.g. Map + WriteToText) instead of to_json.

Example fix

// before
defer.to_json(result, 'out.json')  # result is neither DataFrame nor Series
// after
defer.to_json(result.to_frame(), 'out.json')  # or orient='index' for a Series

proxy = df._expr.proxy()
if not isinstance(proxy, (pd.DataFrame, pd.Series)):
    raise TypeError('to_json requires a deferred DataFrame or Series')

def is_deferred_frame_or_series(df):
    return isinstance(df._expr.proxy(), (pd.DataFrame, pd.Series))

try:
    beam_df.io.to_json(df, path)
except frame_base.WontImplementError:
    df = df.to_frame()
    beam_df.io.to_json(df, path)

When it happens

Trigger: Calling beam.dataframe.io.to_json(df, path, orient=None) where df is a Beam DeferredDataFrame whose _expr.proxy() is neither a pandas DataFrame nor a pandas Series (e.g. an unexpected intermediate expression type).

Common situations: Passing a non-dataframe deferred expression (e.g. a scalar or grouped result) to to_json; constructing deferred frames through unusual transforms that lose the DataFrame/Series proxy; forgetting that Beam's dataframe API implements only a pandas subset.

Understand the failure class

Background: "is not a compatible type" / "cannot merge" errors: when a value's type doesn't match what the library requires — this error's family across 65 libraries.

Related errors

• append() only accepts DeferredDataFrame instances, received
• append() only accepts DeferredSeries instances, received
• Indexing a series with key of type
• Accessing a DeferredSeries with an iterator is sensitive to…
• Accessing an item by an integer key is order sensitive for…

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8977681b02ea9828. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/dataframe/io.py:163

      pd.read_json,
      path,
      args,
      kwargs,
      incremental=kwargs.get('lines', False),
      splitter=_DelimSplitter(b'\n', _DEFAULT_BYTES_CHUNKSIZE) if kwargs.get(
          'lines', False) else None,
      binary=False)


@frame_base.with_docs_from(pd.DataFrame)
def to_json(df, path, orient=None, *args, **kwargs):
  if orient is None:
    if isinstance(df._expr.proxy(), pd.DataFrame):
      orient = 'columns'
    elif isinstance(df._expr.proxy(), pd.Series):
      orient = 'index'
    else:
      raise frame_base.WontImplementError('not dataframes or series')
  kwargs['orient'] = orient
  return _as_pc(df) | _WriteToPandas(
      'to_json',
      path,
      args,
      kwargs,
      incremental=orient in ('index', 'records', 'values'),
      binary=False)


@frame_base.with_docs_from(pd)
def read_html(path, *args, **kwargs):
  return _ReadFromPandas(
      lambda *args, **kwargs: pd.read_html(*args, **kwargs)[0],
      path,
      args,
      kwargs)

View on GitHub (pinned to 12126d8942)