apache/beam · error · IllegalArgumentException

Extra packages were already specified

Error message

Extra packages were already specified

What it means

This IllegalArgumentException is thrown by PythonMap.withExtraPackages when it is called more than once on the same builder-style PythonMap instance. The library enforces a single specification of extra PyPi packages so that the resulting transform configuration is unambiguous. Calling withExtraPackages a second time would either duplicate or conflict with previously set packages, so it is rejected eagerly.

Source

Thrown at sdks/java/extensions/python/src/main/java/org/apache/beam/sdk/extensions/python/transforms/PythonMap.java:79

  public PythonMap<InputT, OutputT> withExpansionService(String expansionService) {
    this.expansionService = expansionService;
    return this;
  }

  /**
   * Specifies any extra packages required by the Python function.
   *
   * <p>This should only be specified when using the default expansion service, i.e. when not using
   * {@link #withExpansionService(String)} to provide an expansion service.
   *
   * <p>The package can either be a PyPi package or the path to a locally available Python package.
   *
   * @param extraPackages a list of PyPi packages. May include the version.
   * @return A {@link PythonMap} with extra packages.
   */
  public PythonMap<InputT, OutputT> withExtraPackages(List<String> extraPackages) {
    if (!this.extraPackages.isEmpty()) {
      throw new IllegalArgumentException("Extra packages were already specified");
    }
    this.extraPackages.addAll(extraPackages);
    return this;
  }

  @Override
  public PCollection<OutputT> expand(PCollection<? extends InputT> input) {
    expansionService = (expansionService != null) ? expansionService : "";
    return (PCollection<OutputT>)
        input.apply(
            PythonExternalTransform.from(pythonTransform, expansionService)
                .withArgs(pythonFunction)
                .withOutputCoder(outputCoder)
                .withExtraPackages(this.extraPackages));
  }
}

View on GitHub (pinned to 12126d8942)

Solutions

1. Call withExtraPackages only once per PythonMap instance; combine all packages into a single List<String> in the first call.
2. If configuration comes from multiple sources, merge the lists before calling withExtraPackages.
3. Create a fresh PythonMap instance if you need different package sets.
4. Refactor the fluent chain so only one method call sets packages.

Example fix

// before
transform = PythonMap.<String, String>of(fn).withExtraPackages(defaultPkgs).withExtraPackages(userPkgs);
// after
List<String> allPkgs = new ArrayList<>(defaultPkgs);
allPkgs.addAll(userPkgs);
transform = PythonMap.<String, String>of(fn).withExtraPackages(allPkgs);

if (!transform.hasExtraPackages()) { // guard or track locally
  transform = transform.withExtraPackages(pkgs);
}

try {
  transform = transform.withExtraPackages(pkgs);
} catch (IllegalArgumentException e) {
  // packages already set; keep existing configuration
}

When it happens

Trigger: Calling withExtraPackages(...) twice on the same PythonMap instance, e.g. chained: map.withExtraPackages(a).withExtraPackages(b), where extraPackages was already populated by a first call.

Common situations: Builder-style pipelines where multiple layers of configuration each call withExtraPackages; merging configuration from two sources (e.g. defaults plus user options) both of which set extra packages; accidental double invocation in fluent chains.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

• Extra packages were already specified
• timerFrequency must be greater than zero
• Could not parse pubsub root url "%s"
• Unrecognized value for stable unique names:
• Unsupported compression type: " + compression

AI-assisted analysis of apache/beam@12126d8942 (2026-09-13). Data as JSON: /api/errors/ddaeedb0f4697481. Report an issue: GitHub.