apache/beam · error · NotImplementedError

f'Unknown provider type: {type} at line {SafeLineLoader.get_

Error message

f'Unknown provider type: {type} at line {SafeLineLoader.get_line(spec)}.'

What it means

provider_from_spec falls through to NotImplementedError when spec['type'] is not present in ExternalProvider._provider_types, i.e. no provider type was registered under that name (registration happens via @ExternalProvider.register_provider_type). The message includes the unknown type and its YAML line.

Source

Thrown at sdks/python/apache_beam/yaml/yaml_provider.py:295

          f'Unexpected parameters in provider of type {type} '
          f'at line {SafeLineLoader.get_line(spec)}: {extra_params}')
    if config.get('version', None) == 'BEAM_VERSION':
      config['version'] = beam_version
    if type in cls._provider_types:
      try:
        constructor = cls._provider_types[type]
        if 'provider_base_path' in inspect.signature(constructor).parameters:
          config['provider_base_path'] = source_path
        result = constructor(urns, **config)
        if not hasattr(result, 'to_json'):
          result.to_json = lambda: spec
        return result
      except Exception as exn:
        raise ValueError(
            f'Unable to instantiate provider of type {type} '
            f'at line {SafeLineLoader.get_line(spec)}: {exn}') from exn
    else:
      raise NotImplementedError(
          f'Unknown provider type: {type} '
          f'at line {SafeLineLoader.get_line(spec)}.')

  @classmethod
  def register_provider_type(cls, type_name):
    def apply(constructor):
      cls._provider_types[type_name] = constructor
      return constructor

    return apply


@ExternalProvider.register_provider_type('javaJar')
def java_jar(urns, provider_base_path, jar: str):
  if not os.path.exists(jar):
    parsed = urllib.parse.urlparse(jar)
    if not parsed.scheme or not parsed.netloc:
      raise ValueError(f'Invalid path or url: {jar}')

View on GitHub (pinned to 12126d8942)

Solutions

  1. Correct the 'type' value to a registered name (javaJar, mavenJar, python, docker, remote, etc.).
  2. Upgrade apache-beam if the provider type was added in a newer release.
  3. Ensure any custom provider module that calls register_provider_type is imported before the pipeline is parsed.

Example fix

# before
- type: java_jar
  config:
    jar: my.jar
# after
- type: javaJar
  config:
    jar: my.jar
Defensive patterns

Strategy: validation

Validate before calling

from apache_beam.yaml.yaml_provider import ExternalProvider
if spec.get('type') not in ExternalProvider._provider_types:
    raise SystemExit(f"Unknown provider type: {spec.get('type')!r}; known: {sorted(ExternalProvider._provider_types)}")

Type guard

def is_known_provider_type(name) -> bool:
    from apache_beam.yaml.yaml_provider import ExternalProvider
    return name in ExternalProvider._provider_types

Try / catch

try:
    provider = ExternalProvider.provider_from_spec(src, spec)
except NotImplementedError as e:
    log.error('%s — check the provider type name and Beam version', e)
    raise

Prevention

When it happens

Trigger: Using type: dockerJar / type: kubernetes or any misspelled/unregistered provider type in a YAML provider spec; using a Beam version that lacks a newer provider type; custom provider plugins not imported before parsing.

Common situations: Typos in provider type names ('python' vs 'Python'); docs referencing provider types from a newer Beam release; third-party providers whose module was never imported.

Understand the failure class

Background: Invalid enum value errors: "Unknown type", "Invalid scope", "must be one of" — when a string is not on the library's allowed list — this error's family across 23 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/8bfbc988e1d7bfc3. Report an issue: GitHub.