{"record":{"id":"db8db54081e06249","repo":"apache/beam","slug":"cannot-infer-a-proxy-because-the-input-pcollection-does-not","errorCode":null,"errorMessage":"Cannot infer a proxy because the input PCollection does not have a schema defined. Please make sure a schema type is specified for the input PCollection, or provide a proxy.","messagePattern":"Cannot infer a proxy because the input PCollection does not have a schema defined\\. Please make sure a schema type is specified for the input PCollection, or provide a proxy\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/dataframe/convert.py","lineNumber":57,"sourceCode":"    pcoll: pvalue.PCollection,\n    proxy: Optional[pd.core.generic.NDFrame] = None,\n    label: Optional[str] = None,\n) -> frame_base.DeferredFrame:\n  \"\"\"Converts a PCollection to a deferred dataframe-like object, which can\n  manipulated with pandas methods like `filter` and `groupby`.\n\n  For example, one might write::\n\n    pcoll = ...\n    df = to_dataframe(pcoll, proxy=...)\n    result = df.groupby('col').sum()\n    pcoll_result = to_pcollection(result)\n\n  A proxy object must be given if the schema for the PCollection is not known.\n  \"\"\"\n  if proxy is None:\n    if pcoll.element_type is None:\n      raise ValueError(\n          \"Cannot infer a proxy because the input PCollection does not have a \"\n          \"schema defined. Please make sure a schema type is specified for \"\n          \"the input PCollection, or provide a proxy.\")\n    # If no proxy is given, assume this is an element-wise schema-aware\n    # PCollection that needs to be batched.\n    if label is None:\n      # Attempt to come up with a reasonable, stable label by retrieving\n      # the name of these variables in the calling context.\n      label = 'BatchElements(%s)' % _var_name(pcoll, 2)\n    proxy = generate_proxy(pcoll.element_type)\n\n    shim_dofn: beam.DoFn\n    if isinstance(proxy, pd.DataFrame):\n      shim_dofn = RowsToDataFrameFn()\n    elif isinstance(proxy, pd.Series):\n      shim_dofn = ElementsToSeriesFn()\n    else:\n      raise AssertionError(\"Unknown proxy type: %s\" % proxy)","sourceCodeStart":39,"sourceCodeEnd":75,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/dataframe/convert.py#L39-L75","documentation":"beam.dataframe.convert.to_dataframe converts a PCollection to a deferred pandas DataFrame. To build the placeholder DataFrame it needs to know the element schema; if the PCollection has element_type None (no schema inferred) and no explicit proxy is supplied, a ValueError is raised telling you to define a schema or pass a proxy.","triggerScenarios":"Calling convert.to_dataframe(pcoll) where pcoll was created from untyped elements (e.g. beam.Create of dicts without element type, JSON-parsed rows) and proxy=None.","commonSituations":"Reading untyped sources (text/JSON/Avro without schema) then using the DataFrame API; pipelines losing type hints through lambdas; migrating pipelines from non-schema PCollections to dataframe transforms.","solutions":["Provide an explicit proxy: convert.to_dataframe(pcoll, proxy=pd.DataFrame({'col': pd.Series([], dtype=...)})).","Give the PCollection an element type/schema (e.g. with_row_type, typed beam.Row, or type hints on the producing DoFn).","Ensure element types aren't erased by lambdas — use named functions or explicit type hints.","If elements are plain dicts, annotate the PCollection with a schema via pc | beam.Map(...).with_output_type(SomeRow)."],"exampleFix":"// before\ndf = convert.to_dataframe(untyped_pcoll)\n// after\nproxy = pd.DataFrame({'col_a': pd.Series(dtype='int64'), 'col_b': pd.Series(dtype=float)})\ndf = convert.to_dataframe(untyped_pcoll, proxy=proxy)","handlingStrategy":"validation","validationCode":"if pcoll.element_type is None and proxy is None:\n    raise ValueError('to_dataframe requires a schema-typed PCollection or an explicit proxy')","typeGuard":null,"tryCatchPattern":"try:\n    df = convert.to_dataframe(pcoll)\nexcept ValueError as e:\n    if 'Cannot infer a proxy' in str(e):\n        df = convert.to_dataframe(pcoll, proxy=build_proxy_from_sample(pcoll))\n    else:\n        raise","preventionTips":["Give producing transforms explicit output types/schemas.","Pass a proxy whenever sources are untyped (JSON/text).","Use with_output_type/with_row_type on DoFns.","Keep element types as named classes, not erasable lambdas."],"tags":["python","pandas","dataframe","schema"],"backgroundTag":"missing-required-argument","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}