apache/beam · error · IllegalArgumentException

Found multiple definitions of aggregate function

Error message

Found multiple definitions of aggregate function ${functionName} in ${jarPath}.

What it means

JavaUdfLoader.loadJar() throws IllegalArgumentException when two aggregate functions in the jar resolve to the same function path key. Duplicate registration in userDefinedAggregateFunctions() is rejected so lookup results are unambiguous. Same mechanism as the scalar-function duplicate check but over the aggregateFunctions map.

Solutions

1. Remove the duplicate aggregate registration or rename one of them.
2. Namespace colliding aggregates under different packages/fully-qualified names.
3. Rebuild and redeploy the corrected jar.
4. Audit userDefinedAggregateFunctions() for key collisions before packaging.

Example fix

// before
return ImmutableMap.of("my.agg", aggA, "my.agg", aggB);
// after
return ImmutableMap.of("my.agg", aggA, "my.agg2", aggB);

Set<List<String>> seen = new HashSet<>();
for (String name : aggregateMap.keySet()) {
  if (!seen.add(ImmutableList.copyOf(name.split("\\.")))) {
    throw new IllegalStateException("Duplicate aggregate fn: " + name);
  }
}

try {
  udfLoader.loadAggregateFunction(fnPath, jarPath);
} catch (IllegalArgumentException e) {
  if (e.getMessage().startsWith("Found multiple definitions of aggregate function")) {
    LOG.error("Duplicate aggregate registration in {}: {}", jarPath, e.getMessage());
  }
  throw e;
}

When it happens

Trigger: loadJar processing a jar whose provider returns two entries in userDefinedAggregateFunctions() that split into the identical List<String> path.

Common situations: The same aggregate registered twice with identical names; merging UDF jars where both define an aggregate with the same fully-qualified name; copy-paste duplication in the provider implementation.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

• Found multiple definitions of scalar function
• No implementation of aggregate function
• Failed to load user-defined aggregate function
• Failed to load user-defined scalar function
• No implementation of scalar function

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/68c6e1fa6ba5ad88. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JavaUdfLoader.java:237

          .userDefinedScalarFunctions()
          .forEach(
              (functionName, implementation) -> {
                List<String> functionPath = ImmutableList.copyOf(functionName.split("\\."));
                if (scalarFunctions.containsKey(functionPath)) {
                  throw new IllegalArgumentException(
                      String.format(
                          "Found multiple definitions of scalar function %s in %s.",
                          functionName, jarPath));
                }
                scalarFunctions.put(functionPath, implementation);
              });
      provider
          .userDefinedAggregateFunctions()
          .forEach(
              (functionName, implementation) -> {
                List<String> functionPath = ImmutableList.copyOf(functionName.split("\\."));
                if (aggregateFunctions.containsKey(functionPath)) {
                  throw new IllegalArgumentException(
                      String.format(
                          "Found multiple definitions of aggregate function %s in %s.",
                          functionName, jarPath));
                }
                aggregateFunctions.put(functionPath, implementation);
              });
    }
    if (providersCount == 0) {
      throw new ProviderNotFoundException(
          String.format(
              "No %s implementation found in %s. Create a class implementing %s and annotate it with @AutoService(%s.class).",
              UdfProvider.class.getSimpleName(),
              jarPath,
              UdfProvider.class.getSimpleName(),
              UdfProvider.class.getSimpleName()));
    }
    LOG.info(
        "Loaded {} implementations of {} from {} with {} scalar function(s).",

View on GitHub (pinned to 12126d8942)