apache/beam · error · ValueError

Missing transform type

Error message

Missing transform type: {identify_object(spec)}

What it means

Scope.create_ptransform builds a PTransform from a YAML spec. Every transform spec must carry a `type` field identifying which transform to instantiate; if `type` is absent, Beam raises this ValueError with the spec's location (identify_object).

Solutions

  1. Add the required `type:` field to the transform spec at the reported location.
  2. Fix YAML indentation so `type` is a sibling key of `name`/`config`, not nested.
  3. If the entry was meant to reference another transform, use the input reference syntax instead of a transform block.
  4. Run the pipeline through Beam's YAML validation/dry-run before submitting.

Example fix

# before
- name: read_rows
  config:
    table: my_table
# after
- name: read_rows
  type: ReadFromBigQuery
  config:
    table: my_table
Defensive patterns

Strategy: validation

Validate before calling

for t in pipeline.get('transforms', []):
    if not isinstance(t, dict) or 'type' not in t:
        raise ValueError(f'Transform missing type: {t.get("name", t)}')

Type guard

def has_type(spec):
    return isinstance(spec, dict) and isinstance(spec.get('type'), str) and bool(spec['type'])

Try / catch

try:
    run_pipeline(spec)
except ValueError as e:
    if 'Missing transform type' in str(e):
        raise UserPipelineError('Each transform entry needs a type: field') from e

Prevention

When it happens

Trigger: A transform entry in the pipeline YAML (or an inline dict passed to create_ptransform) lacking the `type` key — e.g. only `name` and `config` were given, or a malformed nested block.

Common situations: Hand-editing YAML and deleting the `type` line; indentation mistakes that swallow `type` into another mapping; programmatically constructed specs omitting `type`; config-only fragments left as list items.

Understand the failure class

Background: "is required", "must be set", "missing required field": configuration validation errors across open-source libraries — this error's family across 36 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/827eb0a52c0c521e. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/yaml_transform.py:341

        if len(possible_providers) == 1:
          break
        # Go downstream one more step.
        adjacent_transforms = sum(
            [list(self.followers(t)) for t in adjacent_transforms], [])

    return possible_providers[0]

  # A method on scope as providers may be scoped...
  def create_ptransform(self, spec, input_pcolls):
    def maybe_with_resource_hints(transform):
      if 'resource_hints' in spec:
        return transform.with_resource_hints(
            **SafeLineLoader.strip_metadata(spec['resource_hints']))
      else:
        return transform

    if 'type' not in spec:
      raise ValueError(f'Missing transform type: {identify_object(spec)}')

    if spec['type'] == 'composite':

      class _CompositeTransformStub(beam.PTransform):
        @staticmethod
        def expand(pcolls):
          if isinstance(pcolls, beam.PCollection):
            pcolls = {'input': pcolls}
          elif isinstance(pcolls, beam.pvalue.PBegin):
            pcolls = {}

          inner_scope = Scope(
              self.root,
              pcolls,
              spec['transforms'],
              self.providers,
              self.input_providers)
          inner_scope.compute_all()

View on GitHub (pinned to 12126d8942)