apache/beam · error · TypeError

Cannot convert element of type {type(element)} to beam.Row f

Error message

Cannot convert element of type {type(element)} to beam.Row for validation in {label}. Element: {element}

What it means

During output_schema validation, elements of a schemaless PCollection must be converted to beam.Row to match the schema. to_row handles dicts and NamedTuple-like objects; anything else raises this TypeError naming the element type and label.

Source

Thrown at sdks/python/apache_beam/yaml/yaml_transform.py:780

        "PCollection for %s has no schema (element_type=Any). "
        "Converting elements to beam.Row based on provided output_schema.",
        label)
    try:
      # Attempt to confer the schemaless elements into schema-aware beam.Row
      # objects
      beam_schema = json_utils.json_schema_to_beam_schema(clean_schema)
      row_type_constraint = schemas.named_tuple_from_schema(beam_schema)

      def to_row(element):
        """
        Convert a single element into the row type constraint type.
        """
        if isinstance(element, dict):
          return row_type_constraint(**element)
        elif hasattr(element, '_asdict'):  # Handle NamedTuple, beam.Row
          return row_type_constraint(**element._asdict())
        else:
          raise TypeError(
              f"Cannot convert element of type {type(element)} to beam.Row "
              f"for validation in {label}. Element: {element}")

      pcoll = pcoll | f'{label}_ConvertToRow' >> beam.Map(
          to_row).with_output_types(row_type_constraint)
    except Exception as e:
      raise ValueError(
          f"Failed to prepare schemaless PCollection for \
            validation in {label}: {e}") from e

  # Add Validation step downstream of current transform
  return pcoll | label >> Validate(
      schema=clean_schema, error_handling=error_handling_spec)


def expand_composite_transform(spec, scope):
  spec = normalize_inputs_outputs(normalize_source_sink(spec))

View on GitHub (pinned to 12126d8942)

Solutions

1. Have the producing transform emit dicts or beam.Row objects (e.g. use MapToFields to reshape elements).
2. If elements are NamedTuples, they are supported — check that the type actually defines _asdict.
3. Insert an explicit MapToFields step before the transform whose output_schema you want validated.
4. Convert to a schema'd PCollection so the conversion path isn't needed.

Example fix

// before
- type: LogData  # emits plain strings
- type: MyTransform
  config:
    output_schema: {schema: 'id: INTEGER'}
// after
- type: MapToFields
  input: logged
  config:
    id: json_parse(element).id
    output_schema: {schema: 'id: INTEGER'}

def row_convertible(el) -> bool:
    return isinstance(el, dict) or hasattr(el, '_asdict')

def is_row_like(el) -> bool:
    return isinstance(el, dict) or hasattr(el, '_asdict')

try:
    expand_output_schema_transform(spec, outputs, eh)
except (TypeError, ValueError) as e:
    if 'Cannot convert element' in str(e):
        print('Emit dicts or beam.Row from the upstream transform')
    else:
        raise

When it happens

Trigger: Applying output_schema to a PCollection whose elements are plain objects, strings, ints, or custom classes that are neither dict nor have _asdict (NamedTuple/beam.Row), when no explicit schema allows direct validation.

Common situations: Reading untyped/JSON-less data (e.g. raw strings from text) and attaching an output_schema; custom transforms emitting plain Python objects instead of dicts or Rows.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• Unrecognized type_info: {type_info!r}
• WriteToText requires an input schema with exactly one field.
• WriteToText requires an input schema with exactly one field,
• f"Mapping destinations {missing} for {type} are not in the u
• f'test specification {identifier} has unknown attributes {li

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/89e4933e4012d3ca. Report an issue: GitHub.