apache/beam · error · ValueError

Found multiple builders under key

Error message

Found multiple builders under key {key}

What it means

_get_subclass_by_key requires exactly one builder subclass per key. If more than one registered subclass returns the same _builder_key(), the lookup is ambiguous and ValueError is raised to prevent selecting the wrong builder.

Solutions

  1. Find the duplicate subclass with the conflicting _builder_key() and remove or rename it
  2. Give your custom subclass a unique _builder_key() value
  3. Remove duplicate imports of modules defining builder subclasses
  4. Check get_all_subclasses() output to identify the conflicting registrations

Example fix

// before
class MyBuilder(SdkContainerImageBuilder):
  @classmethod
  def _builder_key(cls): return 'local'
// after
class MyBuilder(SdkContainerImageBuilder):
  @classmethod
  def _builder_key(cls): return 'my_custom_local'
Defensive patterns

Strategy: validation

Validate before calling

keys = [s._builder_key() for s in SdkContainerImageBuilder.get_all_subclasses()]
dupes = {k for k in keys if keys.count(k) > 1}
assert not dupes, f'Duplicate builder keys registered: {dupes}'

Type guard

def keys_are_unique() -> bool:
    keys = [s._builder_key() for s in SdkContainerImageBuilder.get_all_subclasses()]
    return len(keys) == len(set(keys))

Try / catch

try:
    builder = SdkContainerImageBuilder._get_subclass_by_key(key)
except ValueError as e:
    if 'Found multiple builders' in str(e):
        raise RuntimeError('Conflicting custom builder registration; remove duplicates') from e
    raise

Prevention

When it happens

Trigger: Two or more SdkContainerImageBuilder subclasses with identical _builder_key() values are importable when build_container_image resolves the key — typically caused by defining or importing duplicate subclasses in the same process.

Common situations: Defining a custom builder subclass that reuses a built-in key like 'local' or 'cloud_build'; importing both a patched and original module; accidental double registration from re-importing under different module paths.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/df4affaaebb2b376. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/runners/portability/sdk_container_builder.py:154

    builder_cls = cls._get_subclass_by_key(container_build_engine)
    builder = builder_cls(pipeline_options)
    return builder._build()

  @classmethod
  def _get_subclass_by_key(cls, key: str) -> type['SdkContainerImageBuilder']:
    available_builders = [
        subclass for subclass in cls.get_all_subclasses()
        if subclass._builder_key() == key
    ]
    if not available_builders:
      available_builder_keys = [
          subclass._builder_key() for subclass in cls.get_all_subclasses()
      ]
      raise ValueError(
          f'Cannot find SDK builder type {key} in '
          f'{available_builder_keys}')
    elif len(available_builders) > 1:
      raise ValueError(f'Found multiple builders under key {key}')
    return available_builders[0]


class _SdkContainerImageLocalBuilder(SdkContainerImageBuilder):
  """SdkContainerLocalBuilder builds the sdk container image with local
  docker."""
  @classmethod
  def _builder_key(cls):
    return 'local_docker'

  def _invoke_docker_build_and_push(self, container_image_name):
    try:
      _LOGGER.info("Building sdk container, this may take a few minutes...")
      now = time.time()
      subprocess.run(['docker', 'build', '.', '-t', container_image_name],
                     check=True,
                     cwd=self._temp_src_dir)
    except subprocess.CalledProcessError as err:

View on GitHub (pinned to 12126d8942)