apache/beam · error · TypeError

cannot check importability of {} instances

Error message

cannot check importability of {} instances

What it means

_should_pickle_by_reference() decides whether an object (module, class, or function) can be pickled by importable reference. It handles only types.ModuleType, function-like, and type objects; anything else (e.g. an instance or partial) reaches the else branch and raises TypeError, since importability cannot be judged for such objects.

Source

Thrown at sdks/python/apache_beam/internal/cloudpickle/cloudpickle.py:354

    """
  if isinstance(obj, types.FunctionType) or issubclass(type(obj), type):
    module_and_name = _lookup_module_and_qualname(obj, name=name, config=config)
    if module_and_name is None:
      return False
    module, name = module_and_name
    return not _is_registered_pickle_by_value(module)

  elif isinstance(obj, types.ModuleType):
    # We assume that sys.modules is primarily used as a cache mechanism for
    # the Python import machinery. Checking if a module has been added in
    # is sys.modules therefore a cheap and simple heuristic to tell us
    # whether we can assume that a given module could be imported by name
    # in another Python process.
    if _is_registered_pickle_by_value(obj):
      return False
    return obj.__name__ in sys.modules
  else:
    raise TypeError(
        "cannot check importability of {} instances".format(type(obj).__name__))


def _lookup_module_and_qualname(obj, name=None, config=DEFAULT_CONFIG):
  if name is None:
    name = getattr(obj, "__qualname__", None)
  if name is None:  # pragma: no cover
    # This used to be needed for Python 2.7 support but is probably not
    # needed anymore. However we keep the __name__ introspection in case
    # users of cloudpickle rely on this old behavior for unknown reasons.
    name = getattr(obj, "__name__", None)

  module_name = _whichmodule(obj, name)

  if module_name is None:
    # In this case, obj.__module__ is None AND obj was not found in any
    # imported module. obj is thus treated as dynamic.
    return None

View on GitHub (pinned to 12126d8942)

Solutions

1. Ensure pickled callables are functions, classes, or modules defined at module level.
2. If hit from Beam pickling, wrap the value so a proper function/class is pickled, or use a DoFn.
3. Check that the object's type is supported before passing it into a pickled pipeline element.
4. Update apache_beam / cloudpickle versions if this arises from a dispatch bug.

Example fix

// before
transform = Map(partial(process, config))  # partial may reach the check
// after
def process_with_config(x, config=config):
    return process(x, config)
transform = Map(process_with_config)

import types, inspect
assert isinstance(obj, (types.ModuleType, type)) or inspect.isfunction(obj), 'object must be module/class/function to pickle by reference'

def reference_pickleable(obj) -> bool:
    import types, inspect
    return isinstance(obj, (types.ModuleType, type)) or inspect.isfunction(obj)

try:
    pickler.dumps(obj)
except TypeError as e:
    if 'cannot check importability' in str(e):
        pickler.dumps(make_module_level_equivalent(obj))
    else:
        raise

When it happens

Trigger: Internal cloudpickle dispatch calling _should_pickle_by_reference with an object that is not a module, type, or function — e.g. pickling functools.partial, a lambda-bound callable wrapper, or a callable class instance that falls through save_global/save_function dispatch.

Common situations: Pickling pipelines containing non-module-level callables (DoFns wrapping callables, Closures over objects); vendored cloudpickle version mismatches where dispatch tables differ.

Understand the failure class

Background: Type mismatch errors: IllegalArgumentException, TypeError and type guards across 150 open-source libraries — this error's family across 150 libraries.

Related errors

• Recursive ID generation detected for {class_def}. The id_gen
• Input should be a module object, got {str(module)} instead
• {module} was not imported correctly, have you used an `impor
• {module} is not registered for pickle by value
• Qual name parts too long

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/0f25cfac05dae4bb. Report an issue: GitHub.