{"record":{"id":"500331b8f7ebd151","repo":"apache/beam","slug":"this-pipeline-runs-with-the-pipeline-option-update","errorCode":null,"errorMessage":"This pipeline runs with the pipeline option --update_compatibility_version=2.67.0 or earlier. When running with this option on SDKs 2.68.0 or later, you must ensure dill==0.3.1.1 is installed.. Dill is not installed.","messagePattern":"This pipeline runs with the pipeline option --update_compatibility_version=2\\.67\\.0 or earlier\\. When running with this option on SDKs 2\\.68\\.0 or later, you must ensure dill==0\\.3\\.1\\.1 is installed\\.\\. Dill is not installed\\.","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/coders/coder_impl.py","lineNumber":370,"sourceCode":"ENUM_TYPE = 103\nNESTED_STATE_TYPE = 104\nDATACLASS_KW_ONLY_TYPE = 105\n\n# Types that can be encoded as iterables, but are not literally\n# lists, etc. due to being lazy.  The actual type is not preserved\n# through encoding, only the elements. This is particularly useful\n# for the value list types created in GroupByKey.\n_ITERABLE_LIKE_TYPES = set()  # type: Set[Type]\n\n\ndef _verify_dill_compat():\n  base_error = (\n      \"This pipeline runs with the pipeline option \"\n      \"--update_compatibility_version=2.67.0 or earlier. \"\n      \"When running with this option on SDKs 2.68.0 or \"\n      \"later, you must ensure dill==0.3.1.1 is installed.\")\n  if not dill:\n    raise RuntimeError(base_error + \". Dill is not installed.\")\n  if dill.__version__ != \"0.3.1.1\":\n    raise RuntimeError(base_error + f\". Found dill version '{dill.__version__}\")\n\n\nclass FastPrimitivesCoderImpl(StreamCoderImpl):\n  \"\"\"For internal use only; no backwards-compatibility guarantees.\"\"\"\n  def __init__(\n      self,\n      fallback_coder_impl,\n      requires_deterministic_step_label=None,\n      force_use_dill=False,\n      use_relative_filepaths=True):\n    self.fallback_coder_impl = fallback_coder_impl\n    self.iterable_coder_impl = IterableCoderImpl(self)\n    self.requires_deterministic_step_label = requires_deterministic_step_label\n    self.warn_deterministic_fallback = True\n    self.force_use_dill = force_use_dill\n    self.use_relative_filepaths = use_relative_filepaths","sourceCodeStart":352,"sourceCodeEnd":388,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/coders/coder_impl.py#L352-L388","documentation":"When a pipeline uses --update_compatibility_version=2.67.0 or earlier, Beam 2.68.0+ requires dill==0.3.1.1 for cross-version pickling compatibility. _verify_dill_compat raises this RuntimeError if the dill module is not installed at all.","triggerScenarios":"Running an update/migration pipeline with --update_compatibility_version<=2.67.0 on SDK 2.68.0+ in an environment where the dill package is absent, and encoding/unpickling a legacy-coded value (encode_type_2_67_0 / _unpickle_type_2_67_0).","commonSituations":"Slimmed runtime environments (Dataflow worker, custom containers, Flink/Spark bundles) where dill was stripped; manually curated dependency sets that dropped dill after upgrading Beam.","solutions":["Install the exact required version: `pip install dill==0.3.1.1`.","Rebuild/publish custom worker containers or dependency bundles including dill==0.3.1.1.","If legacy compatibility is not needed, drop --update_compatibility_version or set it to a newer version so the legacy code path is not used."],"exampleFix":"// before\npip install apache-beam==2.68.0\n// after\npip install apache-beam==2.68.0 dill==0.3.1.1","handlingStrategy":"validation","validationCode":"try:\n    import dill\nexcept ImportError:\n    raise SystemExit('pip install dill==0.3.1.1 before running update-compatibility pipelines')","typeGuard":null,"tryCatchPattern":"try:\n    pipeline.run().wait_until_finish()\nexcept RuntimeError as e:\n    if 'Dill is not installed' in str(e):\n        subprocess.run(['pip', 'install', 'dill==0.3.1.1'], check=True)","preventionTips":["Include dill==0.3.1.1 in requirements/constraints when using --update_compatibility_version<=2.67.0.","Verify custom containers and worker dependencies include dill."],"tags":["python","dill","compatibility","pickling","update"],"backgroundTag":"missing-dependency","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}