apache/beam · error · ValueError

f"Provider file {source_path} must be a list of Providers"

Error message

f"Provider file {source_path} must be a list of Providers"

What it means

load_providers() loads a YAML provider file and expects the top-level YAML document to be a list of provider specifications. If the parsed document is not a list (e.g. a mapping or scalar), a ValueError is raised because parse_providers cannot iterate provider specs.

Source

Thrown at sdks/python/apache_beam/yaml/yaml_provider.py:1652

    return FileSystems.join(FileSystems.split(base)[0], path)


def _read_url_or_filepath(path):
  scheme = urllib.parse.urlparse(path, '').scheme
  if scheme and scheme in urllib.parse.uses_netloc:
    with urllib.request.urlopen(path) as response:
      return response.read()
  else:
    with FileSystems.open(path) as fin:
      return fin.read()


def load_providers(source_path: str) -> Iterable[Provider]:
  from apache_beam.yaml.yaml_transform import SafeLineLoader
  provider_specs = yaml.load(
      _read_url_or_filepath(source_path), Loader=SafeLineLoader)
  if not isinstance(provider_specs, list):
    raise ValueError(f"Provider file {source_path} must be a list of Providers")
  return parse_providers(source_path, provider_specs)


@_as_list
def parse_providers(source_path,
                    provider_specs: Iterable[Mapping]) -> Iterable[Provider]:
  from apache_beam.yaml.yaml_transform import SafeLineLoader
  for provider_spec in provider_specs:
    if 'include' in provider_spec:
      if len(SafeLineLoader.strip_metadata(provider_spec)) != 1:
        raise ValueError(
            f"When using include, it must be the only parameter: "
            f"{provider_spec} "
            f"at {source_path}:{SafeLineLoader.get_line(provider_spec)}")
      include_path = _join_url_or_filepath(
          source_path, provider_spec['include'])
      try:
        yield from load_providers(include_path)

View on GitHub (pinned to 12126d8942)

Solutions

1. Make the YAML file's root a list of provider specs (start with '- type: ...').
2. Remove any top-level wrapper key such as 'providers:' so entries are at the document root.
3. Convert a single provider spec into a one-element list.
4. Validate the file with yaml.safe_load and assert isinstance(result, list) before passing it to Beam YAML.

Example fix

# before (providers.yaml)
providers:
  - type: Sql
    config: {}

# after
- type: Sql
  config: {}

import yaml
specs = yaml.safe_load(open(path))
if not isinstance(specs, list):
    raise ValueError(f'{path}: provider file root must be a list, got {type(specs).__name__}')

def is_provider_list(specs) -> bool:
    return isinstance(specs, list) and all(isinstance(s, dict) for s in specs)

When it happens

Trigger: Calling load_providers(path) or passing --provider_config_file where the YAML file's root is a mapping (e.g. 'providers:' wrapper key) or a single scalar instead of a top-level YAML sequence of provider specs.

Common situations: Users wrap provider definitions under a top-level key like 'providers:', copy a JSON object instead of a list, or merge multiple provider files into a dict.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

• f"When using include, it must be the only parameter: {provid
• f"Error loading providers from {include_path} included at {s
• Node ID cannot be empty
• Edge source and target cannot be empty
• Invalid feature store yaml file provided. Make sure the %s c

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/85d959b49af95e31. Report an issue: GitHub.