apache/beam · warning

Cannot import {constructor} as {fully_qualified_name}.

Error message

Cannot import {constructor} as {fully_qualified_name}.

What it means

When capturing a transform's configuration, Beam attempts to round-trip the constructor by loading it from a fully qualified name (PythonCallableWithSource.load_from_fully_qualified_name) and comparing it to the original wrapper. If the import fails (or the loaded object differs), it warns 'Cannot import <constructor> as <fully_qualified_name>' and returns the transform unannotated, so cross-language/spec-based reconstruction of the transform won't be possible.

Source

Thrown at sdks/python/apache_beam/transforms/ptransform.py:1242

  """Causes instances of this transform to be annotated with their yaml syntax.

  Should only be used for transforms that are fully defined by their constructor
  arguments.
  """
  @wraps(constructor)
  def wrapper(*args, **kwargs):
    transform = constructor(*args, **kwargs)

    fully_qualified_name = (
        f'{constructor.__module__}.{constructor.__qualname__}')
    try:
      imported_constructor = (
          python_callable.PythonCallableWithSource.
          load_from_fully_qualified_name(fully_qualified_name))
      if imported_constructor != wrapper:
        raise ImportError('Different object.')
    except ImportError:
      warnings.warn(f'Cannot import {constructor} as {fully_qualified_name}.')
      return transform

    try:
      config = json.dumps({
          'constructor': fully_qualified_name,
          'args': args,
          'kwargs': kwargs,
      })
    except TypeError as exn:
      warnings.warn(
          f'Cannot serialize arguments for {constructor} as json: {exn}')
      return transform

    original_annotations = transform.annotations
    transform.annotations = lambda: {
        **original_annotations(),
        # These override whatever may have been provided earlier.
        # The outermost call is expected to be the most specific.

View on GitHub (pinned to 12126d8942)

Solutions

1. Move the constructor/callable into an installable module available on both the submitting and worker environments and reference it by its real module path.
2. Ensure the module's import path matches the fully qualified name (avoid __main__; run via python -m or restructure).
3. Pin the same package versions on submit and run environments so the loaded object equals the wrapper.
4. Ignore the warning if spec-based serialization of this transform is not needed.

Example fix

# before (in script run directly)
def my_callable(): ...
# after
# my_pkg/transforms.py
def my_callable(): ...
# then use my_pkg.transforms.my_callable and install my_pkg in workers

import importlib
module, _, name = 'my_pkg.transforms'.rpartition('.')
assert hasattr(importlib.import_module(module), name), 'constructor not importable from workers'

import warnings
with warnings.catch_warnings(record=True) as w:
    warnings.simplefilter('always')
    # build pipeline
    if any('Cannot import' in str(x.message) for x in w):
        print('transform spec annotation skipped; ensure constructor is importable on workers')

When it happens

Trigger: Annotating a ptransform whose callable is defined in a __main__ script, a notebook, or a module not importable in the target environment (different sys.path, package not installed).

Common situations: Notebook/local pipeline definitions submitted to Dataflow or Flink; dynamically defined lambdas/functions; packaging mismatches between submit and worker environments.

Related errors

• `UnknownCoderWrapper` was used to perform an actual encoding
• cannot encode a null Integer
• cannot encode a null Long
• cannot encode a null Short
• cannot encode a null BitSet

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d88de507bb7aad04. Report an issue: GitHub.