apache/beam · error · ValueError

f"Error loading providers from {include_path} included at {s

Error message

f"Error loading providers from {include_path} included at {source_path}:{SafeLineLoader.get_line(provider_spec)}\n{str(exn)}"

What it means

When a provider spec uses 'include', parse_providers recursively calls load_providers on the included file. If that nested load fails for any reason, the exception is wrapped in a ValueError that names the include file, the including file, and the line of the include directive, chained from the original exception.

Source

Thrown at sdks/python/apache_beam/yaml/yaml_provider.py:1673

@_as_list
def parse_providers(source_path,
                    provider_specs: Iterable[Mapping]) -> Iterable[Provider]:
  from apache_beam.yaml.yaml_transform import SafeLineLoader
  for provider_spec in provider_specs:
    if 'include' in provider_spec:
      if len(SafeLineLoader.strip_metadata(provider_spec)) != 1:
        raise ValueError(
            f"When using include, it must be the only parameter: "
            f"{provider_spec} "
            f"at {source_path}:{SafeLineLoader.get_line(provider_spec)}")
      include_path = _join_url_or_filepath(
          source_path, provider_spec['include'])
      try:
        yield from load_providers(include_path)

      except Exception as exn:
        raise ValueError(
            f"Error loading providers from {include_path} included at "
            f"{source_path}:{SafeLineLoader.get_line(provider_spec)}\n" +
            str(exn)) from exn
    else:
      yield ExternalProvider.provider_from_spec(source_path, provider_spec)


def merge_providers(*provider_sets) -> Mapping[str, Iterable[Provider]]:
  result = collections.defaultdict(list)
  for provider_set in provider_sets:
    if isinstance(provider_set, Provider):
      provider = provider_set
      provider_set = {
          transform_type: [provider]
          for transform_type in provider.provided_transforms()
      }
    elif isinstance(provider_set, list):
      provider_set = merge_providers(*provider_set)

View on GitHub (pinned to 12126d8942)

Solutions

1. Read the chained 'Caused by' exception for the root cause.
2. Fix the path in the include directive relative to the including provider file's location.
3. Verify the included file exists, is readable, and has a top-level list of provider specs.
4. Load the included file standalone with load_providers() to isolate the failure.

Example fix

# before
- include: ./providrs.yaml

# after
- include: ./providers.yaml

import os, yaml
inc = spec['include']
full = os.path.join(os.path.dirname(source_path), inc)
if not os.path.exists(full):
    raise FileNotFoundError(f'included provider file missing: {full}')

try:
    yield from load_providers(include_path)
except ValueError as e:
    logging.error('Include failed (check nested cause): %s', e.__cause__)
    raise

When it happens

Trigger: An 'include: path.yaml' directive where the included file is missing, unreadable, malformed YAML, or itself fails provider validation (e.g. not a list).

Common situations: Relative include paths resolved incorrectly against the parent file's location, typos in the include path, or cascading failures when an included file was edited and broke.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

• f"Provider file {source_path} must be a list of Providers"
• f"When using include, it must be the only parameter: {provid
• src and dst files do not exist. src: %s, dst: %s
• Artifacts not found at location: %s when using read_artifact
• Vocabulary file {} not found in artifact location

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/45b69425731840d4. Report an issue: GitHub.