apache/beam · error · IllegalArgumentException

Found multiple definitions of scalar function

Error message

Found multiple definitions of scalar function ${functionName} in ${jarPath}.

What it means

JavaUdfLoader.loadJar() throws IllegalArgumentException when a UdfProvider in the jar registers two scalar functions whose names resolve to the same function path (split on '.'). The loader treats function paths as unique map keys, so duplicate registration is rejected rather than silently overwritten. This detects conflicting definitions shipped in the same jar.

Solutions

Remove or rename the duplicate entry in userDefinedScalarFunctions().
Use distinct fully-qualified names (different package or name) for colliding functions.
Rebuild the jar after de-duplicating and redeploy.
When merging libraries, namespace one under a different package.

Example fix

// before
return ImmutableMap.of("my.fn", fnA, "my.fn", fnB);
// after
return ImmutableMap.of("my.fn", fnA, "my.otherFn", fnB);

Defensive patterns

Strategy: try-catch

Validate before calling

// detect duplicate registration in your own provider before packaging
Set<List<String>> seen = new HashSet<>();
for (String name : providerMap.keySet()) {
  if (!seen.add(ImmutableList.copyOf(name.split("\\.")))) {
    throw new IllegalStateException("Duplicate scalar fn: " + name);
  }
}

Try / catch

try {
  udfLoader.loadScalarFunction(fnPath, jarPath);
} catch (IllegalArgumentException e) {
  if (e.getMessage().startsWith("Found multiple definitions of scalar function")) {
    LOG.error("Duplicate UDF registration in jar {}: {}", jarPath, e.getMessage());
  }
  throw e;
}

Prevention

Never register the same fully-qualified function name twice
When merging UDF libraries, rename colliding functions
Add a build-time test that loads the jar and expects no duplicates
Use constants for function names instead of string literals

When it happens

Trigger: loadJar scanning a jar where userDefinedScalarFunctions() returns two entries mapping to the same List<String> path (e.g. the same fully-qualified name registered twice, or names colliding after '.' splitting).

Common situations: Accidentally registering the same function name twice in the provider map; merging two UDF libraries into one jar where both define the same fully-qualified function name; copy-paste duplication in the provider class.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ea0c328b60c242f0. Report an issue: GitHub.

Appendix: source

Thrown at sdks/java/extensions/sql/src/main/java/org/apache/beam/sdk/extensions/sql/impl/JavaUdfLoader.java:224

      LOG.debug("Using cached function definitions from {}", jarPath);
      return functionCache.get(jarPath);
    }

    ClassLoader classLoader = createClassLoader(jarPath);
    Map<List<String>, ScalarFn> scalarFunctions = new HashMap<>();
    Map<List<String>, AggregateFn> aggregateFunctions = new HashMap<>();
    Iterator<UdfProvider> providers = getUdfProviders(classLoader);
    int providersCount = 0;
    while (providers.hasNext()) {
      providersCount++;
      UdfProvider provider = providers.next();
      provider
          .userDefinedScalarFunctions()
          .forEach(
              (functionName, implementation) -> {
                List<String> functionPath = ImmutableList.copyOf(functionName.split("\\."));
                if (scalarFunctions.containsKey(functionPath)) {
                  throw new IllegalArgumentException(
                      String.format(
                          "Found multiple definitions of scalar function %s in %s.",
                          functionName, jarPath));
                }
                scalarFunctions.put(functionPath, implementation);
              });
      provider
          .userDefinedAggregateFunctions()
          .forEach(
              (functionName, implementation) -> {
                List<String> functionPath = ImmutableList.copyOf(functionName.split("\\."));
                if (aggregateFunctions.containsKey(functionPath)) {
                  throw new IllegalArgumentException(
                      String.format(
                          "Found multiple definitions of aggregate function %s in %s.",
                          functionName, jarPath));
                }
                aggregateFunctions.put(functionPath, implementation);

View on GitHub (pinned to 12126d8942)