apache/beam · error · ValueError

expecting type %s to have arity %d, had arity %d instead

Error message

expecting type %s to have arity %d, had arity %d instead

What it means

In Apache Beam's Python SDK, convert_to_beam_type maps Python native typing constructs to internal Beam type objects via a lookup table of matched entries, each with an expected arity (number of type parameters). This ValueError is raised when a typing construct's number of type arguments does not match the arity recorded for it in the compatibility mapping table. It guards against malformed or unsupported typing expressions such as wrong subscript counts.

Source

Thrown at sdks/python/apache_beam/typehints/native_type_compatibility.py:542

    elif (_match_issubclass(typing.Iterator)(typ) or
          _match_is_exactly_iterable(typ)):
      args = (typehints.TypeVariable('T_co'), )
    else:
      args = (typehints.TypeVariable('T'), ) * arity
  elif matched_entry.arity == -1:
    arity = len_args
  # Counters are special dict types that are implicitly parameterized to
  # [T, int], so we fix cases where they only have one argument to match
  # a more traditional dict hint.
  elif len_args == 1 and _safe_issubclass(getattr(typ, '__origin__', typ),
                                          collections.Counter):
    args = (args[0], int)
    len_args = 2
    arity = matched_entry.arity
  else:
    arity = matched_entry.arity
    if len_args != arity:
      raise ValueError(
          'expecting type %s to have arity %d, had arity %d '
          'instead' % (str(typ), arity, len_args))
  typs = convert_to_beam_types(args)
  if arity == 0:
    # Nullary types (e.g. Any) don't accept empty tuples as arguments.
    return matched_entry.beam_type
  elif arity == 1:
    # Unary types (e.g. Set) don't accept 1-tuples as arguments
    return matched_entry.beam_type[typs[0]]
  else:
    return matched_entry.beam_type[tuple(typs)]


def convert_to_beam_types(args):
  """Convert the given list or dictionary of args to Beam types.

  Args:
    args: Either an iterable of types, or a dictionary where the values are

View on GitHub (pinned to 12126d8942)

Solutions

Fix the type annotation so the number of type parameters matches the construct (e.g. Any takes no parameters: use typing.Any, not typing.Any[x]).
Replace the unsupported typing construct with an equivalent standard generic supported by Beam (List[int], Sequence[str], Tuple[...], etc.).
If converting many types, pre-check the count of __args__ on the typing object before calling convert_to_beam_type and catch ValueError as a fallback.
Check the Beam version's native_type_compatibility mapping table (MATCHED_ENTRIES) to confirm the expected arity for the construct.

Example fix

// before
beam_type = convert_to_beam_type(typing.Any[str])  # arity mismatch
// after
beam_type = convert_to_beam_type(typing.Any)  # nullary, matches expected arity 0

Defensive patterns

Strategy: validation

Validate before calling

def is_safe_for_beam(typ):
    args = typing.get_args(typ)
    if typing.get_origin(typ) is None:
        return True
    # Any and other nullary forms take no args
    if typ is typing.Any:
        return len(args) == 0
    return len(args) in (1, 2)  # typical supported arities

Type guard

def is_supported_typing_generic(typ) -> bool:
    return hasattr(typ, '__origin__') and hasattr(typ, '__args__')

Try / catch

try:
    beam_type = convert_to_beam_type(typ)
except ValueError as e:
    logging.warning('Falling back to Any for type %r: %s', typ, e)
    beam_type = typehints.Any

Prevention

Use standard typing generics (List, Dict, Sequence, Tuple, Union) in pipeline annotations.
Never subscript nullary forms like typing.Any.
Pin and test the Beam SDK version alongside your Python version; typing internals change across versions.
Add unit tests that call convert_to_beam_type on all annotations used in your pipeline.

When it happens

Trigger: Calling convert_to_beam_type (directly or via from_callable, with_output_types, or _extract_tagged_from_type) with a typing generic whose subscription argument count disagrees with the mapping table's arity, e.g. a nullary or multi-arg special form passed with the wrong number of parameters.

Common situations: Annotating DoFn process methods or ParDo with_output_types with unusual/legacy typing constructs (e.g. typing.Sequence with two parameters, or Any subscripted), often after a Python version upgrade changed typing internals.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/c59a8716eaf0e7af. Report an issue: GitHub.