What causes "Module {parts[0]} not found in sys.modules" in apache/beam?

Import the module explicitly before resolving: `import mymodule` then call get_code_from_identifier.

apache/beam · error · AttributeError

Module {parts[0]} not found in sys.modules

Error message

Module {parts[0]} not found in sys.modules

What it means

get_code_from_identifier requires the first dot-separated component of the path to be a module currently present in sys.modules. If the top-level module name is not importable/loaded, it raises AttributeError 'Module {parts[0]} not found in sys.modules'. This ensures identifiers only resolve against modules the current process has actually imported.

Source

Thrown at sdks/python/apache_beam/internal/code_object_pickler.py:439

def get_code_from_identifier(code_object_identifier: str):
  """Returns the code object corresponding to the code object identifier.

  Args:
    code_object_identifier: A string representing the code object identifier.

  Returns:
    The code object.

  Raises:
    ValueError: If the path is empty or invalid.
    AttributeError: If the attribute is not found.
  """
  if not code_object_identifier:
    raise ValueError('Path must not be empty.')
  parts = code_object_identifier.split('.')
  if parts[0] not in sys.modules:
    raise AttributeError(f'Module {parts[0]} not found in sys.modules')
  obj = sys.modules[parts[0]]
  for part in parts[1:]:
    if name_result := _SINGLE_NAME_PATTERN.fullmatch(part):
      obj = _get_code_object_from_single_name_pattern(
          obj, name_result, code_object_identifier)
    elif lambda_with_args_result := _LAMBDA_WITH_ARGS_PATTERN.fullmatch(part):
      obj = _get_code_object_from_lambda_with_args_pattern(
          obj, lambda_with_args_result, code_object_identifier)
    elif lambda_with_hash_result := _LAMBDA_WITH_HASH_PATTERN.fullmatch(part):
      obj = _get_code_object_from_lambda_with_hash_pattern(
          obj, lambda_with_hash_result, code_object_identifier)
    elif default_result := _DEFAULT_PATTERN.fullmatch(part):
      index = int(default_result.group(2))
      if index >= len(obj.__defaults__):
        raise ValueError(
            f'Index {index} is out of bounds for obj.__defaults__'
            f' {len(obj.__defaults__)} in path {code_object_identifier}')
      obj = getattr(obj, '__defaults__')[index]

View on GitHub (pinned to 12126d8942)

Solutions

Import the module explicitly before resolving: `import mymodule` then call get_code_from_identifier.
Ensure the module name in the identifier matches sys.modules (print sys.modules keys to verify).
Avoid pickling functions defined in __main__; move them into an importable installed package with consistent naming.
Install the required package on the worker so the module can be imported.

Example fix

// before
get_code_from_identifier('helpers.transform')  # helpers never imported
// after
import helpers
get_code_from_identifier('helpers.transform')

Defensive patterns

Strategy: type-guard

Validate before calling

import sys
def module_part_loaded(path):
    root = path.split('.', 1)[0]
    return root in sys.modules

Type guard

def is_importable_module_path(path) -> bool:
    import sys
    return isinstance(path, str) and path.split('.', 1)[0] in sys.modules

Try / catch

try:
    code = get_code_from_identifier(path)
except AttributeError as e:
    import importlib
    mod = path.split('.', 1)[0]
    try:
        importlib.import_module(mod)
        code = get_code_from_identifier(path)
    except ImportError:
        raise RuntimeError(f'Module {mod} missing on this environment') from e

Prevention

Import the module before resolving identifiers.
Avoid defining serialized functions in __main__; use installable packages.
Ensure all dependencies are installed on workers (requirements.txt/container image).

When it happens

Trigger: Calling get_code_from_identifier('notimported.func') where 'notimported' was never imported; module imported under a different top-level name (aliasing or package re-rooting); identifier created in __main__ context being resolved where the module has a real package name.

Common situations: Identifiers serialized from a notebook/script (__main__) and replayed in a worker where the module is installed under a different name; missing dependency on the worker; typos in the module part of the path.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/3883c75ace57bee5. Report an issue: GitHub.