apache/beam · error · ValueError

Unknown type or missing provider for type {spec["type"]} for

Error message

Unknown type or missing provider for type {spec["type"]} for {identify_object(spec)}

What it means

During expansion, ensure_transforms_have_providers checks each transform's type against the set of known transforms (those with registered providers, plus 'chain' and 'composite'). An unknown type means either a typo or a valid type whose provider/plugin was not loaded, so expansion cannot proceed and this ValueError is raised.

Source

Thrown at sdks/python/apache_beam/yaml/yaml_transform.py:1331

def apply_phase(phase, spec):
  spec = phase(spec)
  if spec['type'] in {'composite', 'chain'} and 'transforms' in spec:
    spec = dict(
        spec, transforms=[apply_phase(phase, t) for t in spec['transforms']])
  return spec


def preprocess(spec, verbose=False, known_transforms=None):
  if verbose:
    pprint.pprint(spec)

  if known_transforms:
    known_transforms = set(known_transforms).union(['chain', 'composite'])

  def ensure_transforms_have_providers(spec):
    if known_transforms:
      if spec['type'] not in known_transforms:
        raise ValueError(
            'Unknown type or missing provider '
            f'for type {spec["type"]} for {identify_object(spec)}')
    return spec

  def preprocess_languages(spec):
    if spec['type'] in ('AssignTimestamps',
                        'Combine',
                        'Filter',
                        'MapToFields',
                        'Partition'):
      language = spec.get('config', {}).get('language', 'generic')
      new_type = spec['type'] + '-' + language
      if known_transforms and new_type not in known_transforms:
        if language == 'generic':
          raise ValueError(f'Missing language for {identify_object(spec)}')
        else:
          raise ValueError(
              f'Unknown language {language} for {identify_object(spec)}')

View on GitHub (pinned to 12126d8942)

Solutions

  1. Check spelling of spec['type'] against the Beam YAML transform catalog.
  2. Register or pass the required provider (e.g. via providers argument or provider config files) so the type becomes known.
  3. Upgrade apache_beam to a version that includes the transform, or replace it with an equivalent built-in transform.

Example fix

# before (no provider registered)
- type: JdbcRead
  config: {url: ..., driver_class_name: ...}
# after (register provider / use standard transform)
- type: ReadFromJdbc
  config: {url: ..., driver_class_name: ...}
# and run with the jdbc provider config supplied
Defensive patterns

Strategy: validation

Validate before calling

def validate_types(spec, known):
    for t in spec.get('transforms', []):
        if t.get('type') not in known | {'chain', 'composite'}:
            raise ValueError(f"unknown transform type: {t.get('type')}")

Type guard

def type_is_known(t, known):
    return t.get('type') in known or t.get('type') in ('chain', 'composite')

Try / catch

try:
    spec = expand_pipeline(spec, providers=providers)
except ValueError as e:
    if 'Unknown type or missing provider' in str(e):
        raise SystemExit(f"{e} — check spelling or register the provider")
    raise

Prevention

When it happens

Trigger: Calling the pipeline expansion path with known_transforms non-empty and a spec whose spec['type'] is not in that set — e.g. type: ReadFromCsv when no standard/provider for it is registered, or a custom transform provider not passed via providers.

Common situations: Typo in the transform type; using a premium/optional transform whose provider jar/plugin was not supplied; running with a Beam version lacking the transform; forgetting to pass --extraProviderConfig or the providers parameter.

Understand the failure class

Background: UnsupportedOperationException and "is not supported" errors: when a library deliberately refuses a call — this error's family across 30 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/f1f1b34f82a332e1. Report an issue: GitHub.