{"record":{"id":"9ee7b338d4823b54","repo":"apache/beam","slug":"this-pipeline-runs-with-the-pipeline-option-update-coders","errorCode":null,"errorMessage":"This pipeline runs with the pipeline option --update_compatibility_version=2.67.0 or earlier. When running with this option on SDKs 2.68.0 or later, you must ensure dill==0.3.1.1 is installed. Error {e}","messagePattern":"This pipeline runs with the pipeline option --update_compatibility_version=2\\.67\\.0 or earlier\\. When running with this option on SDKs 2\\.68\\.0 or later, you must ensure dill==0\\.3\\.1\\.1 is installed\\. Error (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"sdks/python/apache_beam/coders/coders.py","lineNumber":1034,"sourceCode":"  def value_coder(self):\n    return self\n\n  def to_type_hint(self):\n    return Any\n\n\ndef _should_force_use_dill():\n  from apache_beam.options.pipeline_options_context import get_pipeline_options\n\n  opts = get_pipeline_options()\n  if opts is None or not opts.is_compat_version_prior_to(\"2.68.0\"):\n    return False\n\n  try:\n    import dill\n    assert dill.__version__ == \"0.3.1.1\"\n  except Exception as e:\n    raise RuntimeError(\"This pipeline runs with the pipeline option \" \\\n    \"--update_compatibility_version=2.67.0 or earlier. When running with \" \\\n    \"this option on SDKs 2.68.0 or later, you must ensure dill==0.3.1.1 \" \\\n    f\"is installed. Error {e}\")\n  return True\n\n\ndef _update_compatible_deterministic_fast_primitives_coder(coder, step_label):\n  \"\"\" Returns the update compatible version of DeterministicFastPrimitivesCoder\n   The differences are in how \"special types\" e.g. NamedTuples, Dataclasses are\n   deterministically encoded.\n\n   - In SDK version <= 2.67.0 dill is used to encode \"special types\"\n   - In SDK version 2.68.0 cloudpickle is used to encode \"special types\" with\n   absolute filepaths in code objects and dynamic functions.\n   - In SDK version 2.69.0 cloudpickle is used to encode \"special types\" with\n   relative filepaths in code objects and dynamic functions.\n  \"\"\"\n  if _should_force_use_dill():","sourceCodeStart":1016,"sourceCodeEnd":1052,"githubUrl":"https://github.com/apache/beam/blob/12126d8942aaf848030c478b4c6a28c6af861c66/sdks/python/apache_beam/coders/coders.py#L1016-L1052","documentation":"When running with --update_compatibility_version=2.67.0 or earlier, the SDK must reproduce the exact serialization behavior of older releases, which requires dill 0.3.1.1. If dill is missing or a different version is found, a RuntimeError is raised with the underlying error interpolated.","triggerScenarios":"Launching a pipeline update with pipeline option --update_compatibility_version set to 2.67.0 or lower on SDK 2.68.0+, while the environment has dill absent or != 0.3.1.1.","commonSituations":"Updating an existing streaming pipeline after upgrading the SDK; worker images with a newer dill; forgetting to pin dill==0.3.1.1 in requirements when doing a compatible update.","solutions":["Install the exact version: pip install dill==0.3.1.1 in the launch and worker environments","Pin dill==0.3.1.1 in setup.py/requirements.txt and rebuild custom containers","If exact dill pinning is impossible, drop --update_compatibility_version (requires the update to be compatible with 2.68.0+ semantics)"],"exampleFix":"// before\napache_beam==2.68.0\ndill==0.3.7\n// after\napache_beam==2.68.0\ndill==0.3.1.1","handlingStrategy":"validation","validationCode":"import dill  # raises ImportError early\nassert dill.__version__ == \"0.3.1.1\", f\"need dill==0.3.1.1, got {dill.__version__}\"","typeGuard":null,"tryCatchPattern":"try:\n    run_pipeline(options_with_update_compatibility)\nexcept RuntimeError as e:\n    if 'update_compatibility_version' in str(e):\n        fix_hint('pip install dill==0.3.1.1')\n    raise","preventionTips":["Pin dill==0.3.1.1 whenever using --update_compatibility_version<=2.67.0","Rebuild custom container images with the pinned dill","Verify dill.__version__ in a pre-flight check before submitting the update"],"tags":["python","version-pin","apache-beam","update-compatibility"],"backgroundTag":"dependency-version-incompatible","analyzedSha":"12126d8942aaf848030c478b4c6a28c6af861c66","analyzedAt":"2026-09-13T01:50:10.254Z","contentChangedAt":"2026-09-13T01:50:10.254Z","schemaVersion":2},"datasetVersion":"2026-09-20T03:17:13.778Z"}