{"record":{"id":"e3f82b475a385cca","repo":"apache/beam","slug":"partial-and-use-subprocess-are-mutually-incompatible","errorCode":null,"errorMessage":"partial and use_subprocess are mutually incompatible.","messagePattern":"partial and use_subprocess are mutually incompatible\\.","errorType":"validation","errorClass":"ValueError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/transforms/core.py","lineNumber":2354,"sourceCode":"      self,\n      fn,\n      args,\n      kwargs,\n      main_tag,\n      dead_letter_tag,\n      exc_class,\n      partial,\n      use_subprocess,\n      threshold,\n      threshold_windowing,\n      timeout,\n      error_handler,\n      on_failure_callback,\n      allow_unsafe_userstate_in_process,\n      resource_hints,\n      pardo_type_hints=None):\n    if partial and use_subprocess:\n      raise ValueError('partial and use_subprocess are mutually incompatible.')\n    self._fn = fn\n    self._args = args\n    self._kwargs = kwargs\n    self._main_tag = main_tag\n    self._dead_letter_tag = dead_letter_tag\n    self._exc_class = exc_class\n    self._partial = partial\n    self._use_subprocess = use_subprocess\n    self._threshold = threshold\n    self._threshold_windowing = threshold_windowing\n    self._timeout = timeout\n    self._error_handler = error_handler\n    self._on_failure_callback = on_failure_callback\n    self._allow_unsafe_userstate_in_process = allow_unsafe_userstate_in_process\n    self._resource_hints = resource_hints\n    self._pardo_type_hints = pardo_type_hints\n    self._extra_tags = None\n","sourceCodeStart":2336,"sourceCodeEnd":2372,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/transforms/core.py#L2336-L2372","documentation":"apache_beam raises this ValueError in the DoFn/ParDo constructor when a user requests a 'partial' DoFn wrapper together with 'use_subprocess'. The two options describe contradictory process models (partially-evaluated function vs a function executed in a subprocess pool), so the library refuses to construct such a DoFn.","triggerScenarios":"Calling ParDo(fn, ..., partial=True, use_subprocess=True) or passing equivalent flags through pardo/Map/FlatMap wrappers so that both partial=True and use_subprocess=True reach core.py's DoFn __init__.","commonSituations":"Users combining Beam's subprocess exception-handling feature (added for isolating crashing user code) with pipeline code that also uses partial DoFn evaluation, typically when merging two pipeline snippets with different options.","solutions":["Remove one of the two flags: set use_subprocess=False if you need partial=True","If subprocess isolation is required, drop partial=True and use a full DoFn","Refactor so the partial evaluation and the subprocess execution happen in separate ParDo transforms"],"exampleFix":"// before\nParDo(fn, partial=True, use_subprocess=True)\n// after\nParDo(fn, use_subprocess=True)  # or partial=True without use_subprocess","handlingStrategy":"validation","validationCode":"if isinstance(pardo_kwargs.get('partial'), bool) and isinstance(pardo_kwargs.get('use_subprocess'), bool) and pardo_kwargs['partial'] and pardo_kwargs['use_subprocess']:\n    raise ValueError('partial and use_subprocess cannot both be True')","typeGuard":"def is_valid_pardo_config(kwargs: dict) -> bool:\n    return not (kwargs.get('partial') and kwargs.get('use_subprocess'))","tryCatchPattern":null,"preventionTips":["Encapsulate DoFn option selection in one helper that asserts mutual exclusivity","Add unit tests for your transform-construction helpers covering flag combinations"],"tags":["python","apache-beam","pardo","configuration-conflict"],"backgroundTag":"mutually-exclusive-options","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}