{"record":{"id":"b6f4aee6e515f96f","repo":"apache/beam","slug":"partition-function-by-must-return-a-string-type-not-split-fn","errorCode":null,"errorMessage":"Partition function \"{by}\" must return a string type not {split_fn_output_type}","messagePattern":"Partition function \"(.+?)\" must return a string type not (.+?)","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/yaml/yaml_mapping.py","lineNumber":801,"sourceCode":"        accepted as well, otherwise an error will be raised.\n      outputs: The set of outputs into which this input is being partitioned.\n      unknown_output: (Optional) If set, indicates a destination output for any\n        elements that are not assigned an output listed in the `outputs`\n        parameter.\n      error_handling: (Optional) Whether and how to handle errors during\n        partitioning.\n      language: The language of the `by` expression.\n  \"\"\"\n  split_fn = _as_callable_for_pcoll(pcoll, by, 'by', language)\n  try:\n    split_fn_output_type = trivial_inference.infer_return_type(\n        split_fn, [pcoll.element_type])\n  except (TypeError, ValueError):\n    pass\n  else:\n    if not typehints.is_consistent_with(split_fn_output_type,\n                                        typehints.Optional[str]):\n      raise ValueError(\n          f'Partition function \"{by}\" must return a string type '\n          f'not {split_fn_output_type}')\n  error_output = error_handling['output'] if error_handling else None\n  if error_output in outputs:\n    raise ValueError(\n        f'Error handling output \"{error_output}\" '\n        f'cannot be among the listed outputs {outputs}')\n  T = TypeVar('T')\n\n  def split(element):\n    tag = split_fn(element)\n    if tag is None:\n      tag = unknown_output\n    if not isinstance(tag, str):\n      raise ValueError(\n          f'Returned output name \"{tag}\" of type {type(tag)} '\n          f'from \"{by}\" must be a string.')\n    if tag not in outputs:","sourceCodeStart":783,"sourceCodeEnd":819,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/yaml/yaml_mapping.py#L783-L819","documentation":"The Partition YAML transform validates that the user-supplied partition function (`by`) returns Optional[str] — the output tag name — by type-checking the function's return hint against typehints.Optional[str]. If the inferred output type is inconsistent (e.g. int, bytes, or non-optional str mismatches), the transform raises naming the function and its actual output type.","triggerScenarios":"Configuring a Partition transform whose `by` callable returns a non-string (e.g. an int index) or a non-Optional type, detected via `typehints.is_consistent_with(split_fn_output_type, typehints.Optional[str])` after the split fn is type-inferred against the element type.","commonSituations":"Writing a partition function returning integer partition indexes (like beam.Partition's classic API); returning plain str (not Optional[str]) which may be rejected depending on hint consistency; returning the tag instead of mapping to one of the declared output names.","solutions":["Change the `by` function to return a string matching one of the declared output names.","If the function can fail, return Optional[str] and handle None via the error-handling output.","Convert non-string results (e.g. ints) to strings before returning: return str(tag)."],"exampleFix":"// before\ndef by(element):\n    return 0 if element['x'] < 10 else 1\n// after\ndef by(element):\n    return 'small' if element['x'] < 10 else 'large'\n","handlingStrategy":"type-guard","validationCode":"import typing\ndef by_check(by):\n    hints = typing.get_type_hints(by)\n    ret = hints.get('return')\n    if ret is None or typing.get_origin(ret) not in (typing.Union,) or str not in typing.get_args(ret):\n        raise TypeError('Partition `by` must be annotated -> Optional[str] (a tag name).')","typeGuard":"def returns_tag_string(fn) -> bool:\n    hints = typing.get_type_hints(fn)\n    ret = hints.get('return')\n    return ret is not None and str in typing.get_args(ret) or ret is str","tryCatchPattern":"try:\n    partitioned = partition_transform.expand(pcoll)\nexcept ValueError as e:\n    if 'must return a string type' in str(e):\n        logger.error('Fix `by` return annotation/body: %s', e)\n    raise","preventionTips":["Annotate the partition function explicitly: def by(x) -> typing.Optional[str]","Return output tag names (strings), not integer indexes","Unit-test the `by` function's return values against the declared output list"],"tags":["apache-beam","yaml","partitioning","type-mismatch"],"backgroundTag":"type-mismatch","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}