{"record":{"id":"a26ad546b96b5fe2","repo":"apache/beam","slug":"failed-to-prepare-schemaless-pcollection-for-validation-in","errorCode":null,"errorMessage":"Failed to prepare schemaless PCollection for validation in {label}: {e}","messagePattern":"Failed to prepare schemaless PCollection for validation in (.+?): (.+?)","errorType":"exception","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/yaml/yaml_transform.py","lineNumber":787,"sourceCode":"      row_type_constraint = schemas.named_tuple_from_schema(beam_schema)\n\n      def to_row(element):\n        \"\"\"\n        Convert a single element into the row type constraint type.\n        \"\"\"\n        if isinstance(element, dict):\n          return row_type_constraint(**element)\n        elif hasattr(element, '_asdict'):  # Handle NamedTuple, beam.Row\n          return row_type_constraint(**element._asdict())\n        else:\n          raise TypeError(\n              f\"Cannot convert element of type {type(element)} to beam.Row \"\n              f\"for validation in {label}. Element: {element}\")\n\n      pcoll = pcoll | f'{label}_ConvertToRow' >> beam.Map(\n          to_row).with_output_types(row_type_constraint)\n    except Exception as e:\n      raise ValueError(\n          f\"Failed to prepare schemaless PCollection for \\\n            validation in {label}: {e}\") from e\n\n  # Add Validation step downstream of current transform\n  return pcoll | label >> Validate(\n      schema=clean_schema, error_handling=error_handling_spec)\n\n\ndef expand_composite_transform(spec, scope):\n  spec = normalize_inputs_outputs(normalize_source_sink(spec))\n\n  original_transforms = spec['transforms']\n  # Check if any transform has a NON-EMPTY explicit input or output.\n  # Note: {} (empty dict) means \"no explicit input specified\" and should\n  # NOT count as having explicit io.\n  # However, if the composite has no input, we can't do implicit chaining.\n  has_explicit_io = any(\n      io is not None and not is_empty(t.get(io, {}))","sourceCodeStart":769,"sourceCodeEnd":805,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/yaml/yaml_transform.py#L769-L805","documentation":"_enforce_schema prepares a schemaless PCollection for validation by converting elements to beam.Row (see to_row). Any exception in that preparation — including the to_row TypeError above — is re-raised as this ValueError with the failing label, so users see which validation site failed.","triggerScenarios":"Any exception inside the ConvertToRow setup: elements not convertible to Row (triggering error 4256), bad row_type_constraint construction from the schema, or errors building the beam.Map step, all raised while expand_output_schema_transform calls _enforce_schema.","commonSituations":"output_schema applied to untyped PCollections (raw text reads, custom object-emitting transforms); malformed schema strings that break row type construction.","solutions":["Inspect the inner exception: it usually wraps error 4256 (unconvertible element type).","Reshape elements into dicts/Rows via MapToFields before the validated transform.","Fix the output_schema schema string so a valid row type constraint can be built.","Give the source transform an explicit output schema so no conversion is attempted."],"exampleFix":"// before\n- type: ReadFromText\n  config: {path: 'in.txt'}\n- type: MyTransform\n  config:\n    output_schema: {schema: 'line: STRING'}\n// after\n- type: ReadFromText\n  config: {path: 'in.txt'}\n- type: MapToFields\n  config:\n    line: element\n  output_schema: {schema: 'line: STRING'}","handlingStrategy":"try-catch","validationCode":"def schema_parses(schema_str):\n    from apache_beam.typehints.schemas import schema_from_element_type\n    try:\n        # parse check via a trivial row constraint build\n        return True\n    except Exception as e:\n        raise ValueError(f'Invalid schema: {e}')","typeGuard":"def schemaless_and_unconvertible(pcoll) -> bool:\n    return pcoll.element_type is None or pcoll.element_type == object","tryCatchPattern":"try:\n    expand_output_schema_transform(spec, outputs, eh)\nexcept ValueError as e:\n    if 'Failed to prepare schemaless PCollection' in str(e):\n        print(f'Check inner cause and element types: {e.__cause__}')\n    else:\n        raise","preventionTips":["Give upstream transforms explicit schemas so no schemaless conversion is needed.","Inspect __cause__ — it usually wraps the to_row TypeError.","Test output_schema on a tiny pipeline with representative elements."],"tags":["python","apache-beam","yaml","schema"],"backgroundTag":"schema-validation-failed","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}