apache/beam · error · ValueError

Partition function " " must return a string type not

Error message

Partition function "{by}" must return a string type not {split_fn_output_type}

What it means

The Partition YAML transform validates that the user-supplied partition function (`by`) returns Optional[str] — the output tag name — by type-checking the function's return hint against typehints.Optional[str]. If the inferred output type is inconsistent (e.g. int, bytes, or non-optional str mismatches), the transform raises naming the function and its actual output type.

Solutions

1. Change the `by` function to return a string matching one of the declared output names.
2. If the function can fail, return Optional[str] and handle None via the error-handling output.
3. Convert non-string results (e.g. ints) to strings before returning: return str(tag).

Example fix

// before
def by(element):
    return 0 if element['x'] < 10 else 1
// after
def by(element):
    return 'small' if element['x'] < 10 else 'large'

import typing
def by_check(by):
    hints = typing.get_type_hints(by)
    ret = hints.get('return')
    if ret is None or typing.get_origin(ret) not in (typing.Union,) or str not in typing.get_args(ret):
        raise TypeError('Partition `by` must be annotated -> Optional[str] (a tag name).')

def returns_tag_string(fn) -> bool:
    hints = typing.get_type_hints(fn)
    ret = hints.get('return')
    return ret is not None and str in typing.get_args(ret) or ret is str

try:
    partitioned = partition_transform.expand(pcoll)
except ValueError as e:
    if 'must return a string type' in str(e):
        logger.error('Fix `by` return annotation/body: %s', e)
    raise

When it happens

Trigger: Configuring a Partition transform whose `by` callable returns a non-string (e.g. an int index) or a non-Optional type, detected via `typehints.is_consistent_with(split_fn_output_type, typehints.Optional[str])` after the split fn is type-inferred against the element type.

Common situations: Writing a partition function returning integer partition indexes (like beam.Partition's classic API); returning plain str (not Optional[str]) which may be rejected depending on hint consistency; returning the tag instead of mapping to one of the declared output names.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• 'elements must be a list of elements'
• Error handling output
• Invalid model_handler specification. Expected dict but was
• "RenamingProvider mappings must be dict of transform…
• Returned output name

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/b6f4aee6e515f96f. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/yaml_mapping.py:801

        accepted as well, otherwise an error will be raised.
      outputs: The set of outputs into which this input is being partitioned.
      unknown_output: (Optional) If set, indicates a destination output for any
        elements that are not assigned an output listed in the `outputs`
        parameter.
      error_handling: (Optional) Whether and how to handle errors during
        partitioning.
      language: The language of the `by` expression.
  """
  split_fn = _as_callable_for_pcoll(pcoll, by, 'by', language)
  try:
    split_fn_output_type = trivial_inference.infer_return_type(
        split_fn, [pcoll.element_type])
  except (TypeError, ValueError):
    pass
  else:
    if not typehints.is_consistent_with(split_fn_output_type,
                                        typehints.Optional[str]):
      raise ValueError(
          f'Partition function "{by}" must return a string type '
          f'not {split_fn_output_type}')
  error_output = error_handling['output'] if error_handling else None
  if error_output in outputs:
    raise ValueError(
        f'Error handling output "{error_output}" '
        f'cannot be among the listed outputs {outputs}')
  T = TypeVar('T')

  def split(element):
    tag = split_fn(element)
    if tag is None:
      tag = unknown_output
    if not isinstance(tag, str):
      raise ValueError(
          f'Returned output name "{tag}" of type {type(tag)} '
          f'from "{by}" must be a string.')
    if tag not in outputs:

View on GitHub (pinned to 12126d8942)