apache/beam · warning · ValueError

{module} is not registered for pickle by value

Error message

{module} is not registered for pickle by value

What it means

unregister_pickle_by_value() raises ValueError when the module's __name__ is not in the _PICKLE_BY_VALUE_MODULES set, i.e. the module was never registered (or was already unregistered) for pickle-by-value.

Source

Thrown at sdks/python/apache_beam/internal/cloudpickle/cloudpickle.py:260

  # sys.modules at registering time and assuming that it will still be in
  # there when accessed during pickling. Another alternative would be to
  # store a weakref to the module. Even though cloudpickle does not implement
  # this introspection yet, in order to avoid a possible breaking change
  # later, we still enforce the presence of module inside sys.modules.
  if module.__name__ not in sys.modules:
    raise ValueError(
        f"{module} was not imported correctly, have you used an "
        "`import` statement to access it?")
  _PICKLE_BY_VALUE_MODULES.add(module.__name__)


def unregister_pickle_by_value(module):
  """Unregister that the input module should be pickled by value."""
  if not isinstance(module, types.ModuleType):
    raise ValueError(
        f"Input should be a module object, got {str(module)} instead")
  if module.__name__ not in _PICKLE_BY_VALUE_MODULES:
    raise ValueError(f"{module} is not registered for pickle by value")
  else:
    _PICKLE_BY_VALUE_MODULES.remove(module.__name__)


def list_registry_pickle_by_value():
  return _PICKLE_BY_VALUE_MODULES.copy()


def _is_registered_pickle_by_value(module):
  module_name = module.__name__
  if module_name in _PICKLE_BY_VALUE_MODULES:
    return True
  while True:
    parent_name = module_name.rsplit(".", 1)[0]
    if parent_name == module_name:
      break
    if parent_name in _PICKLE_BY_VALUE_MODULES:
      return True

View on GitHub (pinned to 12126d8942)

Solutions

1. Only call unregister after a successful register_pickle_by_value on the same module.
2. Check membership first: if mymodule.__name__ in list_registry_pickle_by_value(): unregister...
3. Make teardown idempotent by catching ValueError around unregister.
4. Register and unregister in paired try/finally blocks.

Example fix

// before
unregister_pickle_by_value(mymodule)  # may raise if never registered
// after
if mymodule.__name__ in list_registry_pickle_by_value():
    unregister_pickle_by_value(mymodule)

from apache_beam.internal.cloudpickle.cloudpickle import list_registry_pickle_by_value
if mod.__name__ not in list_registry_pickle_by_value():
    return  # nothing to unregister

def is_registered(mod) -> bool:
    from apache_beam.internal.cloudpickle.cloudpickle import list_registry_pickle_by_value
    return mod.__name__ in list_registry_pickle_by_value()

try:
    unregister_pickle_by_value(mod)
except ValueError as e:
    if 'not registered for pickle by value' in str(e):
        pass  # already unregistered; ignore in teardown
    else:
        raise

When it happens

Trigger: Calling unregister_pickle_by_value(mymodule) without a prior register_pickle_by_value(mymodule); calling it twice; registering under a different module (e.g. package submodule vs re-exported alias).

Common situations: Cleanup/teardown in tests that runs even when registration never happened; double teardown; module renamed between register and unregister.

Understand the failure class

Background: "Not found" and "does not exist" errors: why "Task not found", "No such folder", and "Can't find" fire when a lookup comes back empty — this error's family across 14 libraries.

Related errors

• Recursive ID generation detected for {class_def}. The id_gen
• Input should be a module object, got {str(module)} instead
• {module} was not imported correctly, have you used an `impor
• cannot check importability of {} instances
• PCollection not available, please run the pipeline.

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/b7f02ca4bb42a024. Report an issue: GitHub.