apache/beam · error · ValueError

Error handling output

Error message

Error handling output "{error_output}" cannot be among the listed outputs {outputs}

What it means

The Partition transform reserves the error-handling output channel separately from the user's partition outputs. If the `error_handling.output` name (or None-derived default) collides with one of the listed `output` names, outputs would be ambiguous, so construction fails naming the conflicting output tag.

Solutions

1. Rename the error_handling output to a unique name not present in the outputs list (e.g. '_errors').
2. Remove the colliding name from the outputs list.
3. Disable error_handling if a reserved error output is not needed.

Example fix

// before
outputs: [small, large, errors]
error_handling: {output: errors, ...}
// after
outputs: [small, large]
error_handling: {output: _partition_errors, ...}

outputs = ['small', 'large']
error_output = error_handling['output'] if error_handling else None
if error_output in outputs:
    raise ValueError(f'Rename error output; {error_output!r} collides with partition outputs.')

try:
    partitioned = partition_transform.expand(pcoll)
except ValueError as e:
    if 'cannot be among the listed outputs' in str(e):
        error_handling['output'] = error_handling['output'] + '_errors'
    else:
        raise

When it happens

Trigger: Configuring a Partition transform with `error_handling: {output: bad}` where 'bad' is also listed among the transform's `output` names; `error_output in outputs` is true during validation in _Partition.

Common situations: Reusing the same tag name for both a valid partition and the error channel; copy-pasting output lists that already include an 'errors'/'invalid' tag while also enabling error_handling with that same name.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

• Can only drop fields if append is true.
• Exploding unknown field
• Input schema has multiple fields
• Invalid feature store yaml file provided. Make sure the
• Only expressions allowed in SQL at

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/7076a1acb33ead02. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/yaml_mapping.py:806

      error_handling: (Optional) Whether and how to handle errors during
        partitioning.
      language: The language of the `by` expression.
  """
  split_fn = _as_callable_for_pcoll(pcoll, by, 'by', language)
  try:
    split_fn_output_type = trivial_inference.infer_return_type(
        split_fn, [pcoll.element_type])
  except (TypeError, ValueError):
    pass
  else:
    if not typehints.is_consistent_with(split_fn_output_type,
                                        typehints.Optional[str]):
      raise ValueError(
          f'Partition function "{by}" must return a string type '
          f'not {split_fn_output_type}')
  error_output = error_handling['output'] if error_handling else None
  if error_output in outputs:
    raise ValueError(
        f'Error handling output "{error_output}" '
        f'cannot be among the listed outputs {outputs}')
  T = TypeVar('T')

  def split(element):
    tag = split_fn(element)
    if tag is None:
      tag = unknown_output
    if not isinstance(tag, str):
      raise ValueError(
          f'Returned output name "{tag}" of type {type(tag)} '
          f'from "{by}" must be a string.')
    if tag not in outputs:
      if unknown_output:
        tag = unknown_output
      else:
        raise ValueError(f'Unknown output name "{tag}" from {by}')
    return beam.pvalue.TaggedOutput(tag, element)

View on GitHub (pinned to 12126d8942)