apache/beam · error · TypeError

Could not determine schema for type hints

Error message

Could not determine schema for type hints {element_types!r}: Inconsistent names: {name_set}

What it means

union_schema_type() zips the fields of all element types and requires that the fields at each position share exactly one name. If the names at a position differ across element types, it raises TypeError with the offending name set. This ensures the unified union schema has unambiguous field names.

Solutions

Rename fields so every element type in the Union uses identical field names in the same order.
Align the types with a single shared base NamedTuple/Row schema and use that in the annotation.
Map each type to a common representation with a DoFn/map before schema inference.

Example fix

// before
class A(NamedTuple):
  id: int

class B(NamedTuple):
  key: int

# Union[A, B] -> Inconsistent names: {'id', 'key'}

// after
class A(NamedTuple):
  id: int

class B(NamedTuple):
  id: int

Defensive patterns

Strategy: validation

Validate before calling

def check_same_field_names(types):
    names = [set(t._fields) for t in types]
    if any(n != names[0] for n in names):
        raise TypeError("Union element types must share identical field names")

Type guard

def same_field_names(types):
    return bool(types) and all(getattr(t, '_fields', None) == getattr(types[0], '_fields', None) for t in types)

Try / catch

try:
    schema = union_schema_type(element_types)
except TypeError as e:
    if "Inconsistent names" in str(e):
        schema = named_tuple_from_schema(common_schema)  # explicit mapping
    else:
        raise

Prevention

Use identical field names across record types participating in a Union.
Centralize record type definitions in one module instead of duplicating them.
Validate union schemas in CI before deploying pipeline changes.

When it happens

Trigger: Calling union_schema_type() (or schema coercion of a Union type hint) where element types have the same field count but different field names at the same position, e.g. Union[Row(a=int), Row(b=int)] or Union[NamedTuple(id=int), NamedTuple(key=int)].

Common situations: Renaming a field in one copy of a NamedTuple but not another; hand-writing Union annotations over structurally similar but differently named record types; combining types from two libraries that model the same concept with different field names.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/decc354d6cf5cab6. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/typehints/schemas.py:739

def union_schema_type(element_types):
  """Returns a schema whose fields are the union of each corresponding field.

  element_types must be a set of schema-aware types whose fields have the
  same naming and ordering.
  """
  named_fields_and_types = []
  for t in element_types:
    n = named_fields_from_element_type(t)
    if named_fields_and_types and len(named_fields_and_types[-1]) != len(n):
      raise TypeError("element types has different number of fields")
    named_fields_and_types.append(n)

  union_fields_and_types = []
  for field in zip(*named_fields_and_types):
    names, types = zip(*field)
    name_set = set(names)
    if len(name_set) != 1:
      raise TypeError(
          f"Could not determine schema for type hints {element_types!r}: "
          f"Inconsistent names: {name_set}")
    union_fields_and_types.append(
        (next(iter(name_set)), typehints.Union[types]))
  return named_tuple_from_schema(named_fields_to_schema(union_fields_and_types))


class _Ephemeral:
  """Helper class for wrapping unpicklable objects."""
  def __init__(self, obj):
    self.obj = obj

  def __reduce__(self):
    return _Ephemeral, (None, )


# Registry of typings for a schema by UUID
class LogicalTypeRegistry(object):

View on GitHub (pinned to 12126d8942)