apache/beam · error · RuntimeError

This pipeline runs with the pipeline option…

Error message

This pipeline runs with the pipeline option --update_compatibility_version=2.67.0 or earlier. When running with this option on SDKs 2.68.0 or later, you must ensure dill==0.3.1.1 is installed. Error {e}

What it means

When running with --update_compatibility_version=2.67.0 or earlier, the SDK must reproduce the exact serialization behavior of older releases, which requires dill 0.3.1.1. If dill is missing or a different version is found, a RuntimeError is raised with the underlying error interpolated.

Solutions

  1. Install the exact version: pip install dill==0.3.1.1 in the launch and worker environments
  2. Pin dill==0.3.1.1 in setup.py/requirements.txt and rebuild custom containers
  3. If exact dill pinning is impossible, drop --update_compatibility_version (requires the update to be compatible with 2.68.0+ semantics)

Example fix

// before
apache_beam==2.68.0
dill==0.3.7
// after
apache_beam==2.68.0
dill==0.3.1.1
Defensive patterns

Strategy: validation

Validate before calling

import dill  # raises ImportError early
assert dill.__version__ == "0.3.1.1", f"need dill==0.3.1.1, got {dill.__version__}"

Try / catch

try:
    run_pipeline(options_with_update_compatibility)
except RuntimeError as e:
    if 'update_compatibility_version' in str(e):
        fix_hint('pip install dill==0.3.1.1')
    raise

Prevention

When it happens

Trigger: Launching a pipeline update with pipeline option --update_compatibility_version set to 2.67.0 or lower on SDK 2.68.0+, while the environment has dill absent or != 0.3.1.1.

Common situations: Updating an existing streaming pipeline after upgrading the SDK; worker images with a newer dill; forgetting to pin dill==0.3.1.1 in requirements when doing a compatible update.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9ee7b338d4823b54. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/coders/coders.py:1034

  def value_coder(self):
    return self

  def to_type_hint(self):
    return Any


def _should_force_use_dill():
  from apache_beam.options.pipeline_options_context import get_pipeline_options

  opts = get_pipeline_options()
  if opts is None or not opts.is_compat_version_prior_to("2.68.0"):
    return False

  try:
    import dill
    assert dill.__version__ == "0.3.1.1"
  except Exception as e:
    raise RuntimeError("This pipeline runs with the pipeline option " \
    "--update_compatibility_version=2.67.0 or earlier. When running with " \
    "this option on SDKs 2.68.0 or later, you must ensure dill==0.3.1.1 " \
    f"is installed. Error {e}")
  return True


def _update_compatible_deterministic_fast_primitives_coder(coder, step_label):
  """ Returns the update compatible version of DeterministicFastPrimitivesCoder
   The differences are in how "special types" e.g. NamedTuples, Dataclasses are
   deterministically encoded.

   - In SDK version <= 2.67.0 dill is used to encode "special types"
   - In SDK version 2.68.0 cloudpickle is used to encode "special types" with
   absolute filepaths in code objects and dynamic functions.
   - In SDK version 2.69.0 cloudpickle is used to encode "special types" with
   relative filepaths in code objects and dynamic functions.
  """
  if _should_force_use_dill():

View on GitHub (pinned to 12126d8942)