apache/beam · error · TypeError

Proxy '{proxy}' has unsupported type '{type(proxy)}'

Error message

Proxy '{proxy}' has unsupported type '{type(proxy)}'

What it means

During unbatching (to_pcollection of deferred dataframes back to elements, via maybe_unbatch/_make_unbatched_pcoll), the proxy object must be a DeferredDataFrame or DeferredSeries. Any other proxy type (e.g. a plain pandas DataFrame or another wrapper) is unsupported and raises TypeError.

Source

Thrown at sdks/python/apache_beam/dataframe/convert.py:129

    label += " with indexes"

  if label not in UNBATCHED_CACHE:
    proxy = expr.proxy()
    shim_dofn: beam.DoFn
    if isinstance(proxy, pd.DataFrame):
      shim_dofn = DataFrameToRowsFn(proxy, include_indexes)
    elif isinstance(proxy, pd.Series):
      if include_indexes:
        warnings.warn(
            "Pipeline is converting a DeferredSeries to PCollection "
            "with include_indexes=True. Note that this parameter is "
            "_not_ respected for DeferredSeries conversion. To "
            "include the index with your data, produce a"
            "DeferredDataFrame instead.")

      shim_dofn = SeriesToElementsFn(proxy)
    else:
      raise TypeError(f"Proxy '{proxy}' has unsupported type '{type(proxy)}'")

    UNBATCHED_CACHE[label] = pc | label >> beam.ParDo(shim_dofn)

  # Note unbatched cache is keyed by the expression id as well as parameters
  # for the unbatching (i.e. include_indexes)
  return UNBATCHED_CACHE[label]


class DataFrameToRowsFn(beam.DoFn):
  def __init__(self, proxy, include_indexes):
    self._proxy = proxy
    self._include_indexes = include_indexes

  @beam.DoFn.yields_batches
  def process(self, element: pd.DataFrame) -> Iterable[pd.DataFrame]:
    yield element

  def infer_output_type(self, input_element_type):

View on GitHub (pinned to 12126d8942)

Solutions

1. Pass a DeferredDataFrame/DeferredSeries proxy — get it via expressions.PlaceholderExpression + frame_base.DeferredFrame.wrap, or omit proxy and ensure the PCollection has a schema.
2. If you have a plain pd.DataFrame, wrap it: frame_base.DeferredFrame.wrap(expressions.PlaceholderExpression(df.iloc[:0], ...)).
3. Check the proxy's type before calling and normalize it to DeferredBase.
4. Avoid passing concrete pandas objects; they are only accepted for to_pcollection's non-deferred inputs, not as proxies.

Example fix

// before
df = convert.to_dataframe(pcoll, proxy=pd.DataFrame(columns=['a', 'b']))
// after
proxy = frame_base.DeferredFrame.wrap(
    expressions.PlaceholderExpression(pd.DataFrame(columns=['a', 'b'])))
df = convert.to_dataframe(pcoll, proxy=proxy)

from apache_beam.dataframe import frame_base
if proxy is not None and not isinstance(proxy, frame_base.DeferredBase):
    raise TypeError('proxy must be a DeferredDataFrame/DeferredSeries')

def is_deferred_frame(obj) -> bool:
    from apache_beam.dataframe import frame_base
    return isinstance(obj, frame_base.DeferredBase)

try:
    out = convert.to_pcollection(df, proxy=proxy)
except TypeError as e:
    if 'unsupported type' in str(e):
        out = convert.to_pcollection(df, proxy=wrap_as_deferred(proxy))
    else:
        raise

When it happens

Trigger: Passing proxy= to convert.to_dataframe/to_pcollection with a raw pd.DataFrame/pd.Series instead of its deferred counterpart, or a proxy of an unrelated type; mixing plain pandas objects into the dataframe-on-Beam API.

Common situations: Users caching a plain pandas placeholder from outside the deferred session; upgraded code paths where a proxy constructed earlier as concrete pandas is reused; custom integrations building their own proxies.

Understand the failure class

Background: "is not a compatible type" / "cannot merge" errors: when a value's type doesn't match what the library requires — this error's family across 65 libraries.

Related errors

• Cannot infer a proxy because the input PCollection does not
• concat(ignore_index)
• concat(levels)
• Encountered unknown type {other!r}
• Proxy '{proxy}' has unsupported type '{type(proxy)}'

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/59609712dcbedca0. Report an issue: GitHub.