apache/beam · error · ValueError

Qual name parts too long

Error message

Qual name parts too long

What it means

Beam's code_object_pickler reconstructs code objects by walking __qualname__ parts. In _search_function, when the current attribute is __code__, the qual-name path must terminate; if more parts remain, it raises ValueError('Qual name parts too long'), meaning the identifier path does not correspond to a valid function-code nesting.

Source

Thrown at sdks/python/apache_beam/internal/code_object_pickler.py:216

def _search_function(
    callable: types.FunctionType,
    node: types.FunctionType,
    qual_name_parts: list[str]):
  """Searches a function to create a code object identifier.

  Args:
    callable: The callable object to search for.
    node: The function to search within.
    qual_name_parts: The list of qual name parts.

  Returns:
    The code object identifier, or None if not found.
  """
  first_part = qual_name_parts[0]
  if (node.__code__ == callable.__code__):
    if len(qual_name_parts) > 1:
      raise ValueError('Qual name parts too long')
    return '__code__'
  # If first part is '<locals>' then the code object is in a local variable
  # so we should add __code__ to the path to indicate that we are entering
  # the code object of the function.
  if first_part == '<locals>':
    return _extend_path(
        '__code__', _search(callable, node.__code__, qual_name_parts))


def _search_code(
    callable: types.FunctionType,
    node: types.CodeType,
    qual_name_parts: list[str]):
  """Searches a code object to create a code object identifier.

  Args:
    callable: The callable to search for.
    node: The code object to search within.

View on GitHub (pinned to 12126d8942)

Solutions

1. Define the pickled callables at module level instead of deep nesting.
2. Avoid exec/compile-generated code with synthetic qualnames in pickled transforms.
3. Catch ValueError and fall back to standard cloudpickle (dill/cloudpickle pickling) for that callable.
4. If caused by the pickler itself, report/upgrade apache_beam version.

Example fix

// before
class Outer:
    def make(self):
        def inner(x): return x  # deep nested qualname path
        return inner
Map(Outer().make())
// after
def inner(x): return x  # module-level
Map(inner)

if len(fn.__qualname__.split('.')) > 6:
    raise ValueError('qualname too deep for code-object pickling; hoist to module level')

def has_simple_qualname(fn) -> bool:
    return getattr(fn, '__qualname__', '').count('.') <= 2

try:
    pickled = beam_pickler.dumps(fn)
except ValueError as e:
    if 'Qual name parts too long' in str(e):
        pickled = cloudpickle.dumps(fn)
    else:
        raise

When it happens

Trigger: Resolving an identifier whose qual name has extra segments after __code__ — i.e. a malformed/ambiguous identifier produced for a nested or unusual callable during pickling of a pipeline's callables.

Common situations: Pickling deeply nested local functions or classes where generated qual-name paths don't match the real object structure; dynamic code objects (exec/compile) with synthetic qualnames.

Understand the failure class

Background: "invalid id" errors: invalid identifier format — why libraries reject IDs before lookup, and how to fix them — this error's family across 37 libraries.

Related errors

• Invalid pattern for single name: {name_result.group(0)}
• cannot check importability of {} instances
• Could not find code object with path: {path}
• Path must not be empty.
• Module {parts[0]} not found in sys.modules

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8dd5efcd4d9fab7e. Report an issue: GitHub.