What causes "Could not find enum descriptor: {full_name}" in apache/beam?

Import the Python module generated from the .proto file before loading, so its descriptors register in the pool.

apache/beam · error · ImportError

Could not find enum descriptor: {full_name}

Error message

Could not find enum descriptor: {full_name}

What it means

_reconstruct_enum_descriptor() scans loaded protobuf descriptor pools for an enum descriptor matching the given full_name and raises ImportError when none matches. This happens during unpickling of protobuf EnumDescriptor objects that Beam's pickler serialized by name.

Source

Thrown at sdks/python/apache_beam/internal/cloudpickle_pickler.py:138

def _reconstruct_enum_descriptor(full_name):
  for _, module in list(sys.modules.items()):
    if not hasattr(module, 'DESCRIPTOR'):
      continue

    if hasattr(module.DESCRIPTOR, 'enum_types_by_name'):
      for (_, enum_desc) in module.DESCRIPTOR.enum_types_by_name.items():
        if enum_desc.full_name == full_name:
          return enum_desc

    for _, attr_value in vars(module).items():
      if not hasattr(attr_value, 'DESCRIPTOR'):
        continue

      if hasattr(attr_value.DESCRIPTOR, 'enum_types_by_name'):
        for (_, enum_desc) in attr_value.DESCRIPTOR.enum_types_by_name.items():
          if enum_desc.full_name == full_name:
            return enum_desc
  raise ImportError(f'Could not find enum descriptor: {full_name}')


def _pickle_enum_descriptor(obj):
  full_name = obj.full_name
  return _reconstruct_enum_descriptor, (full_name, )


def dumps(
    o,
    enable_trace=True,
    use_zlib=False,
    enable_best_effort_determinism=False,
    enable_stable_code_identifier_pickling=False,
    config: cloudpickle.CloudPickleConfig = DEFAULT_CONFIG) -> bytes:
  """For internal use only; no backwards-compatibility guarantees."""
  s = _dumps(
      o,
      enable_best_effort_determinism,

View on GitHub (pinned to 12126d8942)

Solutions

Import the Python module generated from the .proto file before loading, so its descriptors register in the pool.
Restore/revert the enum's original full_name, or re-save the session with current protos.
Ensure the worker environment includes the same proto definitions version used when pickling.
Search the descriptor pool for the enum and update references if it was renamed.

Example fix

// before
with beam.Pipeline() as p:  # enum proto never imported on worker
    ...
// after
import myprotos_pb2  # registers enum descriptors
load_session('session.pkl')

Defensive patterns

Strategy: try-catch

Validate before calling

from google.protobuf import descriptor_pool
assert full_name in descriptor_pool.Default().EnumDescriptors() or True  # probe pool before load

Type guard

def enum_in_pool(full_name: str) -> bool:
    try:
        from google.protobuf import descriptor_pool
        descriptor_pool.Default().FindEnumTypeByName(full_name)
        return True
    except KeyError:
        return False

Try / catch

try:
    load_session(path)
except ImportError as e:
    if 'Could not find enum descriptor' in str(e):
        import suspect_protos_pb2  # register missing descriptors and retry
        load_session(path)
    else:
        raise

Prevention

Import all generated proto modules before unpickling.
Keep proto schemas in sync between save and load environments.
Use identical Beam/proto versions in workers.

When it happens

Trigger: Unpickling a session/pipeline whose pickled enum descriptor's full_name no longer exists in the current .proto definitions — the proto was renamed, removed, or the proto file was never imported so its descriptors are not in the pool.

Common situations: Proto schema changed between save and load; worker image lacks the proto module containing the enum; proto package name changed; deserializing old saved sessions (load_session) against newer code.

Understand the failure class

Background: "cannot parse invalid wire-format data", "cannot unmarshal", "failed unmarshalling": protobuf unmarshal errors explained — this error's family across 10 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/9597f2651b1dc103. Report an issue: GitHub.