apache/beam · warning · ValueError

Duplicate name at

Error message

Duplicate name at {identify_object(spec)}: {name}

What it means

Scope.unique_name assigns each PTransform a unique graph label. If a name was already seen, strictness >= 2 raises 'Duplicate name'; at lower strictness it silently disambiguates by appending @line. This error appears when strict duplicate checking is enabled and two transforms share a name.

Solutions

1. Rename one of the duplicate transforms so all names are unique.
2. Add line numbers/ids programmatically when generating specs.
3. Lower naming strictness if @line-suffixed duplicates are acceptable.
4. Audit generated pipelines for template loops reusing a fixed name.

Example fix

# before
- name: write_output
  type: WriteToJson
  ...
- name: write_output
  type: WriteToJson
  ...
# after
- name: write_output_json
  type: WriteToJson
  ...
- name: write_output_csv
  type: WriteToJson
  ...

import collections
names = [t.get('name', t['type']) for t in pipeline['transforms']]
dups = [n for n, c in collections.Counter(names).items() if c > 1]
if dups:
    raise ValueError(f'Duplicate transform names: {dups}')

def names_are_unique(specs):
    seen = set()
    for s in specs:
        n = s.get('name', s.get('type'))
        if n in seen:
            return False
        seen.add(n)
    return True

try:
    run_pipeline(spec, strict_naming=True)
except ValueError as e:
    if 'Duplicate name' in str(e):
        raise UserPipelineError('Rename one of the duplicated transforms') from e

When it happens

Trigger: unique_name (invoked during create_ptransform naming) receives a spec whose `name` (or fallback type label) already exists in `self._seen_names`, with strictness >= 2 — e.g. validation passes that enforce strict naming.

Common situations: Copy-pasted YAML blocks with identical names; generated pipelines looping without unique suffixes; relying on the lenient @line rename while running strict validation; a transform name colliding with a type label.

Understand the failure class

Background: "invalid id" errors: invalid identifier format — why libraries reject IDs before lookup, and how to fix them — this error's family across 37 libraries.

Related errors

• Config for transform at
• f'Ambiguous output at line
• f'Ambiguous transform at line
• f'Unknown output at line : only has outputs
• Invalid transform specification at

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/5fa16234218edd80. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/yaml_transform.py:469

      else:
        msg = str(exn)
      raise ValueError(
          f'Invalid transform specification at {identify_object(spec)}: {msg}'
      ) from exn

  def unique_name(self, spec, ptransform, strictness=0):
    if 'name' in spec:
      name = spec['name']
      strictness += 1
    elif ('ExternalTransform' not in ptransform.label and
          not ptransform.label.startswith('_')):
      # The label may have interesting information.
      name = ptransform.label
    else:
      name = spec['type']
    if name in self._seen_names:
      if strictness >= 2:
        raise ValueError(f'Duplicate name at {identify_object(spec)}: {name}')
      else:
        name = f'{name}@{SafeLineLoader.get_line(spec)}'
    self._seen_names.add(name)
    return name


def expand_transform(spec, scope):
  if 'type' not in spec:
    raise TypeError(
        f'Missing type parameter for transform at {identify_object(spec)}')
  type = spec['type']
  if type == 'composite':
    return expand_composite_transform(spec, scope)
  else:
    return expand_leaf_transform(spec, scope)


def expand_leaf_transform(spec, scope):

View on GitHub (pinned to 12126d8942)