apache/beam · error · ValueError

'No available provider for type %r at

Error message

'No available provider for type %r at %s%s'

What it means

Scope.best_provider selects which provider (built-in Python, Java, expansion service, etc.) executes a transform type. If providers for the type exist in the registry but none are currently `available()` (and no single usable candidate can be chosen), Beam raises this ValueError, appending a list of which providers were found but unavailable and why.

Solutions

1. Read the appended 'providers were found but not available' messages and fix the stated cause (install Java, start the expansion service, install jars).
2. Install the required extra dependencies or configure the provider's environment (Java SDK, expansion service address).
3. Use an alternative transform type with an available built-in provider.
4. Register a custom provider in the pipeline spec pointing at a reachable expansion service.

Example fix

# before (yaml, no java environment)
- type: JdbcWrite
  ...
# after: make a provider available, then keep the spec
# start: java -jar beam-sdks-java-io-expansion-service.jar 44441
- type: JdbcWrite
  ...
# with a java provider configured/reachable

providers = scope.providers.get(spec_type, [])
if not any(p.available() for p in providers):
    raise EnvironmentError(f'No available provider for {spec_type}; check Java/expansion service')

try:
    result = run_pipeline(yaml_spec)
except ValueError as e:
    if 'No available provider' in str(e):
        print(e)  # message lists which providers were unavailable and why
        raise RuntimeError('Install Java SDK / start expansion service') from e

When it happens

Trigger: Using a transform type whose only providers report `available() == False` — e.g. Java-provider-only transforms without a Java environment, external transforms requiring an unreachable expansion service, or providers needing missing dependencies/extra dependencies.

Common situations: Running a Java cross-language transform (Kafka/JDBC) in an environment without Java; expansion service endpoint down; missing jars/Docker; restricted runtime without optional dependencies.

Understand the failure class

Background: "not installed", "pip install", "required for": how missing-dependency errors surface across open-source libraries — this error's family across 34 libraries.

Related errors

• Ambiguous expression type (perhaps missing quoting?)
• Ambiguous expression type (perhaps missing quoting?)
• Can only use expressions on a schema'd input.
• CombineFn spec missing type
• Config for transform at

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/60eb567d539bf2e0. Report an issue: GitHub.

Appendix: source

Thrown at sdks/python/apache_beam/yaml/yaml_transform.py:270

      spec = self._transforms_by_uuid[self.get_transform_id(t)]
    possible_providers = []
    unavailable_provider_messages = []
    for p in self.providers[spec['type']]:
      is_available = p.available()
      if is_available:
        possible_providers.append(p)
      else:
        reason = getattr(is_available, 'reason', 'no reason given')
        unavailable_provider_messages.append(
            f'{p.__class__.__name__} ({reason})')
    if not possible_providers:
      if unavailable_provider_messages:
        unavailable_provider_message = (
            '\nThe following providers were found but not available: ' +
            '\n'.join(unavailable_provider_messages))
      else:
        unavailable_provider_message = ''
      raise ValueError(
          'No available provider for type %r at %s%s' %
          (spec['type'], identify_object(spec), unavailable_provider_message))
    # From here on, we have the invariant that possible_providers is not empty.

    # Only one possible provider, no need to rank further.
    if len(possible_providers) == 1:
      return possible_providers[0]

    def best_matches(
        possible_providers: Iterable[yaml_provider.Provider],
        adjacent_provider_options: Iterable[Iterable[yaml_provider.Provider]]
    ) -> list[yaml_provider.Provider]:
      """Given a set of possible providers, and a set of providers for each
      adjacent transform, returns the top possible providers as ranked by
      affinity to the adjacent transforms' providers.
      """
      providers_by_score = collections.defaultdict(list)
      for p in possible_providers:

View on GitHub (pinned to 12126d8942)