apache/beam · error · ProviderNotFoundException

No UdfProvider implementation found in ${jarPath}. Create a

Error message

No UdfProvider implementation found in ${jarPath}. Create a class implementing UdfProvider and annotate it with @AutoService(UdfProvider.class).

What it means

JavaUdfLoader.loadJar scans a user-supplied JAR for a class implementing the UdfProvider interface (discovered via ServiceLoader / @AutoService metadata) to register its UDFs and aggregate functions. When the JAR contains no such provider (providersCount == 0), the loader throws ProviderNotFoundException because there is nothing to load. It is a user-JAR contract violation, not an internal failure.

Source

Thrown at sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JavaUdfLoader.java:246

                }
                scalarFunctions.put(functionPath, implementation);
              });
      provider
          .userDefinedAggregateFunctions()
          .forEach(
              (functionName, implementation) -> {
                List<String> functionPath = ImmutableList.copyOf(functionName.split("\\."));
                if (aggregateFunctions.containsKey(functionPath)) {
                  throw new IllegalArgumentException(
                      String.format(
                          "Found multiple definitions of aggregate function %s in %s.",
                          functionName, jarPath));
                }
                aggregateFunctions.put(functionPath, implementation);
              });
    }
    if (providersCount == 0) {
      throw new ProviderNotFoundException(
          String.format(
              "No %s implementation found in %s. Create a class implementing %s and annotate it with @AutoService(%s.class).",
              UdfProvider.class.getSimpleName(),
              jarPath,
              UdfProvider.class.getSimpleName(),
              UdfProvider.class.getSimpleName()));
    }
    LOG.info(
        "Loaded {} implementations of {} from {} with {} scalar function(s).",
        providersCount,
        UdfProvider.class.getSimpleName(),
        jarPath,
        scalarFunctions.size());
    FunctionDefinitions userFunctionDefinitions =
        FunctionDefinitions.newBuilder()
            .setScalarFunctions(ImmutableMap.copyOf(scalarFunctions))
            .setAggregateFunctions(ImmutableMap.copyOf(aggregateFunctions))
            .build();

View on GitHub (pinned to 12126d8942)

Solutions

Create a public class implementing org.apache.beam.sdk.extensions.sql.udf.UdfProvider and annotate it with com.google.auto.service.AutoService(UdfProvider.class).
Verify META-INF/services/org.apache.beam.sdk.extensions.sql.udf.UdfProvider exists inside the JAR listing the implementation class (unzip -l jarPath).
Ensure the auto-service annotation processor runs: keep the com.google.auto.service:auto-service dependency in the 'compile' scope (processor) of the UDF module.
If using maven-shade-plugin, add the ServicesResourceTransformer so service files from dependencies are merged.
Double-check the jarPath passed to loadJar points at the artifact containing the provider class.

Example fix

// before
public class MyUdfs { /* no UdfProvider */ }
// after
@AutoService(UdfProvider.class)
public class MyUdfs implements UdfProvider {
  @Override
  public Collection<Udf> getUserDefinedFunctions() {
    return Collections.singletonList(new MyFn());
  }
}

Defensive patterns

Strategy: validation

Validate before calling

boolean hasProvider(java.util.jar.JarFile jar) throws java.io.IOException {
  return jar.getEntry("META-INF/services/org.apache.beam.sdk.extensions.sql.udf.UdfProvider") != null;
}

Prevention

Always annotate the provider class with @AutoService(UdfProvider.class) and keep auto-service as an annotation processor dependency.
Add an integration test that loads the built JAR via JavaUdfLoader before release.
Verify service files survive shading by checking the final artifact's META-INF/services directory.

When it happens

Trigger: Calling JavaUdfLoader.loadJar(jarPath, functionNames) (via functionDefinitions) with a JAR that contains zero classes implementing UdfProvider registered under META-INF/services/org.apache.beam...UdfProvider — e.g. built without @AutoService(UdfProvider.class) annotation processing or an explicit services resource file.

Common situations: Forgetting to add the auto-service dependency/annotation processor so META-INF/services is never generated; annotating the wrong class or an inner class; building a fat JAR that strips service files (e.g. shade merging without ServicesResourceTransformer); pointing at a data/dependency JAR instead of the UDF JAR.

Understand the failure class

Background: "not installed", "pip install", "required for": how missing-dependency errors surface across open-source libraries — this error's family across 34 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/d1a0cfe1a283d79c. Report an issue: GitHub.