apache/beam · error · RuntimeException

Failed to load user-defined aggregate function ${functionFul

Error message

Failed to load user-defined aggregate function ${functionFullName} from ${jarPath}

What it means

JavaUdfLoader.loadAggregateFunction() converts IOException thrown by loadJar(jarPath) into RuntimeException 'Failed to load user-defined aggregate function %s from %s'. The aggregate function lookup never occurs because the jar itself could not be opened, read, or its classes loaded.

Source

Thrown at sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JavaUdfLoader.java:114

  public AggregateFn loadAggregateFunction(List<String> functionPath, String jarPath) {
    String functionFullName = String.join(".", functionPath);
    try {
      FunctionDefinitions functionDefinitions = loadJar(jarPath);
      if (!functionDefinitions.aggregateFunctions().containsKey(functionPath)) {
        throw new IllegalArgumentException(
            String.format(
                "No implementation of aggregate function %s found in %s.%n"
                    + " 1. Create a class implementing %s and annotate it with @AutoService(%s.class).%n"
                    + " 2. Add function %s to the class's userDefinedAggregateFunctions implementation.",
                functionFullName,
                jarPath,
                UdfProvider.class.getSimpleName(),
                UdfProvider.class.getSimpleName(),
                functionFullName));
      }
      return functionDefinitions.aggregateFunctions().get(functionPath);
    } catch (IOException e) {
      throw new RuntimeException(
          String.format(
              "Failed to load user-defined aggregate function %s from %s",
              functionFullName, jarPath),
          e);
    }
  }

  /**
   * Creates a temporary local copy of the file at {@code inputPath}, and returns a handle to the
   * local copy.
   */
  private File downloadFile(String inputPath, String mimeType) throws IOException {
    Preconditions.checkArgument(!inputPath.isEmpty(), "Path cannot be empty.");

    // Issue warning when downloading from public repositories
    if (inputPath.startsWith("http://") || inputPath.startsWith("https://")) {
      if (inputPath.contains("repo.maven.apache.org")
          || inputPath.contains("repo1.maven.org")

View on GitHub (pinned to 12126d8942)

Solutions

Confirm the jar exists and is readable at jarPath from the process executing the query.
Validate the jar with jar tf and confirm the UdfProvider service registration is inside.
Fix staging/upload of the jar to the runner environment.
Read the wrapped IOException cause for the precise failure.

Example fix

// before
loader.loadAggregateFunction(ImmutableList.of("myagg"), "missing.jar");
// after (validate first)
if (!new File(jarPath).exists()) throw new IllegalArgumentException("missing jar");
loader.loadAggregateFunction(ImmutableList.of("myagg"), jarPath);

Defensive patterns

Strategy: try-catch

Validate before calling

File jar = new File(jarPath);
if (!jar.isFile() || !jar.canRead()) {
  throw new IllegalArgumentException("UDF jar missing or unreadable: " + jarPath);
}

Try / catch

try {
  AggregateFn fn = udfLoader.loadAggregateFunction(fnPath, jarPath);
} catch (RuntimeException e) {
  if (e.getMessage().startsWith("Failed to load user-defined aggregate function")) {
    LOG.error("Could not read jar {}: {}", jarPath, e.getCause().getMessage());
  }
  throw e;
}

Prevention

Verify jar presence/readability on workers before query execution
Detect corrupted jars during deployment with a load check
Avoid duplicate/conflicting classpaths that break classloading
Log and monitor the IOException cause for infrastructure issues

When it happens

Trigger: loadAggregateFunction invoked with a jarPath that is missing, unreadable, corrupt/not a jar, or whose provider classes fail to load (IOException in loadJar).

Common situations: Jar not present on the worker node; path typo; permission problems; corrupted upload; classloader conflicts inside the jar.

Understand the failure class

Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/02bd60d41f29269a. Report an issue: GitHub.