apache/beam · error · ValueError

Number of components does not match number of coders.

Error message

Number of components does not match number of coders.

What it means

TupleCoderImpl.encode_to_stream extracts the components of a tuple-like value and requires exactly one component coder per component. If the value's component count differs from the number of coders the TupleCoder was constructed with, Beam raises ValueError because there is no defined way to encode the mismatched shape.

Source

Thrown at sdks/python/apache_beam/coders/coder_impl.py:1161

  """For internal use only; no backwards-compatibility guarantees.

  CoderImpl for coders that are comprised of several component coders."""
  def __init__(self, coder_impls):
    for c in coder_impls:
      assert isinstance(c, CoderImpl), c
    self._coder_impls = tuple(coder_impls)

  def _extract_components(self, value):
    raise NotImplementedError

  def _construct_from_components(self, components):
    raise NotImplementedError

  def encode_to_stream(self, value, out, nested):
    # type: (Any, create_OutputStream, bool) -> None
    values = self._extract_components(value)
    if len(self._coder_impls) != len(values):
      raise ValueError('Number of components does not match number of coders.')
    for i in range(0, len(self._coder_impls)):
      c = self._coder_impls[i]  # type cast
      c.encode_to_stream(
          values[i], out, nested or i + 1 < len(self._coder_impls))

  def decode_from_stream(self, in_stream, nested):
    # type: (create_InputStream, bool) -> Any
    return self._construct_from_components([
        c.decode_from_stream(
            in_stream, nested or i + 1 < len(self._coder_impls))
        for i, c in enumerate(self._coder_impls)
    ])

  def estimate_size(self, value, nested=False):
    # type: (Any, bool) -> int

    """Estimates the encoded size of the given value, in bytes."""
    # TODO(ccy): This ignores sizes of observable components.

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure the values emitted match the arity the TupleCoder was built for (same number of components)
2. Recreate the TupleCoder from the current value type/schema so coder count matches components
3. If the schema changed, re-generate coders (let Beam infer from the new type hint) instead of reusing stale ones
4. For variable-length data use a sequence/list coder rather than a fixed-arity TupleCoder

Example fix

// before
encoder = TupleCoder((VarIntCoder(), StrUtf8Coder()))  # 2 coders
encoder.encode((1, 'a', True))  # 3 components -> ValueError
// after
encoder = TupleCoder((VarIntCoder(), StrUtf8Coder(), BooleanCoder()))
encoder.encode((1, 'a', True))

components = value if isinstance(value, tuple) else tuple(extract_components(value))
assert len(components) == len(coder_impls), \
    f'expected {len(coder_impls)} components, got {len(components)}'

def matches_arity(value, coder) -> bool:
    return len(tuple(value)) == len(coder._coder_impls)

try:
    coder.encode(value)
except ValueError as e:
    if 'components' in str(e):
        raise TypeError(f'{value!r} does not match coder arity: {e}') from e
    raise

When it happens

Trigger: Encoding a value whose _extract_components() yields a different number of items than the coder list given to TupleCoderImpl._construct_from / TupleCoder — e.g., encoding a 3-tuple with a coder built for 2 fields, or a typed row whose fields changed since the coder was created.

Common situations: Schema/PCollection type changed (field added/removed) between writer and reader; reusing a cached or deserialized coder against new data shapes; passing plain tuples of varying length into a DoFn output coded by a fixed TupleCoder.

Related errors

• Encountered unexpected value for null indicator: '%s'
• Integer value '{value}' is out of the encodable range for Va
• Cannot estimate size for integer value '{value}'. Value is o
• Cannot read state-written iterable without state reader.
• Invalid PaneInfoEncoding: %s

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/75947b94ea58417c. Report an issue: GitHub.