apache/beam · error · ValueError

Circular reference detected: Transform

Error message

Circular reference detected: Transform {name} references itself as input in {identify_object(spec)}

What it means

Beam YAML's transform-reference validation raises this when a pipeline transform lists itself (by name or transform type) as one of its own inputs, in validate_transform_references. Self-referencing inputs create an unresolvable dependency cycle, so the graph can never be evaluated and validation fails fast.

Solutions

Edit the YAML so the transform's input points to the correct upstream transform, not itself.
If the transform should read from the pipeline source, set input to the actual source transform name.
Rename the transform so its name no longer collides with the intended input reference, then update the input.

Example fix

// before
- name: MyTransform
  type: MapToFields
  input: MyTransform
  config: {...}
// after
- name: MyTransform
  type: MapToFields
  input: ReadFromPubSub
  config: {...}

Defensive patterns

Strategy: validation

Validate before calling

def check_no_self_reference(transform):
    inputs = transform.get('input', {}).get('input', [])
    inputs = [inputs] if isinstance(inputs, str) else (inputs or [])
    for ref in inputs:
        if ref in (transform.get('name'), transform.get('type')):
            raise ValueError(f"{transform.get('name')} references itself via input '{ref}'")

Type guard

def is_self_reference(t):
    ins = t.get('input', {}).get('input', [])
    ins = [ins] if isinstance(ins, str) else (ins or [])
    return any(r in (t.get('name'), t.get('type')) for r in ins)

Try / catch

try:
    expand_pipeline(yaml_spec)
except ValueError as e:
    if 'Circular reference detected' in str(e):
        print(f'Fix self-referencing input in YAML: {e}')

Prevention

Lint YAML pipelines for name/input collisions before submission.
Keep transform names distinct from their upstream source names.
Validate pipelines with Beam's YAML validation API in CI.

When it happens

Trigger: A YAML transform spec has an 'input' (or input.input list) whose value equals the transform's own 'name' or its 'type' string; raised during graph expansion of the pipeline spec.

Common situations: Copy-pasted YAML where a transform's input was accidentally left pointing at itself; templated/generated YAML that substitutes the same name into both name and input fields; typos in the input field naming the wrong transform.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/e579ced3c2729b20. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/yaml_transform.py:1363

        if language == 'generic':
          raise ValueError(f'Missing language for {identify_object(spec)}')
        else:
          raise ValueError(
              f'Unknown language {language} for {identify_object(spec)}')
      return dict(spec, type=new_type, name=spec.get('name', spec['type']))
    else:
      return spec

  def validate_transform_references(spec):
    name = spec.get('name', '')
    transform_type = spec.get('type')
    inputs = spec.get('input').get('input', [])

    if not is_empty(inputs):
      input_values = [inputs] if isinstance(inputs, str) else inputs
      for input_value in input_values:
        if input_value in (name, transform_type):
          raise ValueError(
              f"Circular reference detected: Transform {name} "
              f"references itself as input in {identify_object(spec)}")

    return spec

  for phase in [
      ensure_transforms_have_types,
      normalize_mapping,
      normalize_combine,
      preprocess_languages,
      ensure_transforms_have_providers,
      preprocess_source_sink,
      preprocess_chain,
      tag_explicit_inputs,
      normalize_inputs_outputs,
      validate_transform_references,
      preprocess_flattened_inputs,
      ensure_errors_consumed,

View on GitHub (pinned to 12126d8942)