{"record":{"id":"c8a0d9a6b153e9fc","repo":"apache/beam","slug":"dofn-self-r-yields-element-from-both-process-and-process","errorCode":null,"errorMessage":"DoFn {self!r} yields element from both process and process_batch, but they have mismatched output typehints:\n process: {process_type_hints.output_types}\n process_batch: {process_batch_type_hints.output_types}","messagePattern":"DoFn (.+?) yields element from both process and process_batch, but they have mismatched output typehints:\n process: (.+?)\n process_batch: (.+?)","errorType":"validation","errorClass":"TypeError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/transforms/core.py","lineNumber":814,"sourceCode":"      # process() produces batches, don't use it's output typehint\n      process_type_hints = process_type_hints.with_output_types_from(\n          typehints.decorators.IOTypeHints.empty())\n\n    if self._process_batch_yields_elements:\n      # process_batch() produces elements, *do* use it's output typehint\n\n      # First access the typehint\n      process_batch_type_hints = typehints.decorators.IOTypeHints.from_callable(\n          self.process_batch) or typehints.decorators.IOTypeHints.empty()\n\n      # Then we deconflict with the typehint from process, if it exists\n      if (process_batch_type_hints.output_types\n          != typehints.decorators.IOTypeHints.empty().output_types):\n        if (process_type_hints.output_types\n            != typehints.decorators.IOTypeHints.empty().output_types and\n            process_batch_type_hints.output_types\n            != process_type_hints.output_types):\n          raise TypeError(\n              f\"DoFn {self!r} yields element from both process and \"\n              \"process_batch, but they have mismatched output typehints:\\n\"\n              f\" process: {process_type_hints.output_types}\\n\"\n              f\" process_batch: {process_batch_type_hints.output_types}\")\n\n        process_type_hints = process_type_hints.with_output_types_from(\n            process_batch_type_hints)\n\n    try:\n      process_type_hints = process_type_hints.strip_iterable()\n    except ValueError as e:\n      raise ValueError('Return value not iterable: %s: %s' % (self, e))\n    process_type_hints = process_type_hints.extract_tagged_outputs()\n\n    # Prefer class decorator type hints for backwards compatibility.\n    return get_type_hints(self.__class__).with_defaults(process_type_hints)\n\n  # TODO(sourabhbajaj): Do we want to remove the responsibility of these from","sourceCodeStart":796,"sourceCodeEnd":832,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/transforms/core.py#L796-L832","documentation":"When a DoFn defines both `process` and `process_batch` and both declare output type hints, Beam requires them to agree (core.py:814). Mismatched output typehints make the pipeline's inferred output type ambiguous, so default_type_hints raises this TypeError.","triggerScenarios":"A DoFn subclass where `process` has e.g. @with_output_types(Iterable[str]) while `process_batch` is annotated/typed as returning batches of ints, and both output hint sets are non-empty and unequal.","commonSituations":"Migrating a DoFn from element-wise `process` to batched `process_batch` while leaving old type hints on `process`; annotations added incrementally by different contributors.","solutions":["Make the output type hints of `process` and `process_batch` consistent (same inner element type).","Delete the now-redundant `process` method if you only use `process_batch`.","Remove explicit output hints from one of the methods so only one source of truth exists."],"exampleFix":"# before\nclass MyDoFn(DoFn):\n    def process(self, x) -> Iterable[str]: ...\n    def process_batch(self, batch) -> Iterator[List[int]]: ...\n\n# after\nclass MyDoFn(DoFn):\n    def process(self, x) -> Iterator[str]: ...\n    def process_batch(self, batch) -> Iterator[List[str]]: ...","handlingStrategy":"validation","validationCode":"import inspect\nfrom typing import get_type_hints\ndef check_matching_output_hints(dofn_cls):\n    if hasattr(dofn_cls, 'process') and hasattr(dofn_cls, 'process_batch'):\n        p = get_type_hints(dofn_cls.process).get('return')\n        pb = get_type_hints(dofn_cls.process_batch).get('return')\n        if p is not None and pb is not None and p != pb:\n            raise TypeError(f'process ({p}) and process_batch ({pb}) output hints differ')","typeGuard":"def has_consistent_hints(cls) -> bool:\n    if not (hasattr(cls, 'process') and hasattr(cls, 'process_batch')):\n        return True\n    from typing import get_type_hints\n    return get_type_hints(cls.process).get('return') in (None, get_type_hints(cls.process_batch).get('return'))","tryCatchPattern":"try:\n    hints = dofn.default_type_hints()\nexcept TypeError as e:\n    if 'mismatched output typehints' in str(e):\n        logger.error('Align process/process_batch output hints: %s', e)\n    raise","preventionTips":["Keep only one of process/process_batch per DoFn when possible","When migrating to batched processing, delete old process methods and their hints","Write a smoke test that instantiates the DoFn and reads default_type_hints()"],"tags":["python","apache-beam","type-hints","typeerror"],"backgroundTag":"type-mismatch","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}