apache/beam · error · ValueError

Failed to decode schema due to an issue with Field proto

Error message

Failed to decode schema due to an issue with Field proto:

{text_format.MessageToString(field)}

What it means

`named_tuple_from_schema` converts each Field proto to a Python type; if any field's type conversion raises ValueError, it is re-raised with the full field proto text appended, so the developer can see exactly which field proto failed. This is a wrapping diagnostic for errors like unknown atomic types or type_info values.

Solutions

Read the embedded field proto in the message to find the offending field
Fix or drop that field from the schema, or convert it to a supported type
Upgrade apache-beam so the field's type is supported

Defensive patterns

Strategy: try-catch

Validate before calling

for f in schema.fields:
  try:
    converter.typing_from_runner_api(f.type)
  except ValueError as e:
    raise ValueError(f'bad field {f.name}: {e}')

Try / catch

try:
  Row = converter.named_tuple_from_schema(schema)
except ValueError as e:
  log.error(str(e))  # message contains the offending Field proto
  raise

Prevention

Inspect the embedded Field proto text in the error to locate the bad field
Validate schemas at pipeline-construction time, not at runtime hydration

When it happens

Trigger: Calling `named_tuple_from_schema` (directly or via union_schema_type / _hydrate_namedtuple_instance) on a Schema proto containing at least one Field whose type cannot be converted to a Python typing.

Common situations: Cross-language schemas with unsupported field types; stale protos from newer SDK versions; corrupted serialized schemas.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f8c2224b4f77ba77. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/typehints/schemas.py:641

          fieldtype_proto.iterable_type.element_type)]

    else:
      raise ValueError(f"Unrecognized type_info: {type_info!r}")

  def named_tuple_from_schema(self, schema: schema_pb2.Schema) -> type:
    from apache_beam import coders

    type_name = 'BeamSchema_{}'.format(schema.id.replace('-', '_'))

    subfields = []
    descriptions = {}
    for field in schema.fields:
      try:
        field_py_type = self.typing_from_runner_api(field.type)
        if isinstance(field_py_type, row_type.RowTypeConstraint):
          field_py_type = field_py_type.user_type
      except ValueError as e:
        raise ValueError(
            "Failed to decode schema due to an issue with Field proto:\n\n"
            f"{text_format.MessageToString(field)}") from e

      descriptions[field.name] = field.description
      subfields.append((field.name, field_py_type))

    if schema.id in self.schema_registry.by_id:
      user_type = self.schema_registry.by_id[schema.id][0]
    else:
      user_type = NamedTuple(type_name, subfields)

      # Define a reduce function, otherwise these types can't be pickled
      # (See BEAM-9574)
      setattr(
          user_type,
          '__reduce__',
          _named_tuple_reduce_method(schema.SerializeToString()))
      setattr(user_type, "_field_descriptions", descriptions)

View on GitHub (pinned to 12126d8942)