apache/hadoop · error · IllegalArgumentException

Duplicate option ${simpleName}

Error message

Duplicate option ${simpleName}

What it means

HandleOpt.getOpt(Class, HandleOpt...) scans for an option assignable to the requested class and throws IllegalArgumentException('Duplicate option <SimpleName>') on a second match. Used when opening files by handle (FileSystem/FileContext open with handle options); note it uses isAssignableFrom, so two subclassed options of the same category also collide.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/Options.java:454

    /**
     * Utility method to extract a HandleOpt from the set provided.
     * @param c Target class
     * @param opt List of options
     * @param <T> Type constraint for exact match
     * @throws IllegalArgumentException If more than one matching type is found.
     * @return An option assignable from the specified type or null if either
     * opt is null or a suitable match is not found.
     */
    public static <T extends HandleOpt> Optional<T> getOpt(
        Class<T> c, HandleOpt... opt) {
      if (null == opt) {
        return Optional.empty();
      }
      T ret = null;
      for (HandleOpt o : opt) {
        if (c.isAssignableFrom(o.getClass())) {
          if (ret != null) {
            throw new IllegalArgumentException("Duplicate option "
                + c.getSimpleName());
          }

          @SuppressWarnings("unchecked")
          T tmp = (T) o;
          ret = tmp;
        }
      }
      return Optional.ofNullable(ret);
    }

    /**
     * Option storing standard constraints on data.
     */
    public static class Data extends HandleOpt {
      private final boolean allowChanged;
      Data(boolean allowChanged) {
        this.allowChanged = allowChanged;

View on GitHub (pinned to 2add963021)

Solutions

1. Pass exactly one option per category (one read-consistency opt, one write opt)
2. Dedupe by assignable category before invoking open(path, opts...)

Example fix

// before
fs.open(p, HandleOpt.dataLocality(), customReadOpt);
// after
fs.open(p, HandleOpt.dataLocality());   // keep exactly one option per category

int readOpts = 0;
for (HandleOpt o : opts) {
  if (HandleOpt.ReadOpt.class.isAssignableFrom(o.getClass())) readOpts++;
}
if (readOpts > 1) throw new IllegalArgumentException("duplicate read handle option");

When it happens

Trigger: Passing two handle options assignable to the same base class in one open call — e.g. HandleOpt.dataLocality() plus another HandleOpt.ReadOpt subclass (both count as read options).

Common situations: Wrapper APIs stacking their own handle option on top of user-supplied ones; copying option examples into a call that already has a category covered.

Related errors

• Key ${name} already exists in ${this}
• multiple opts varargs: ${clazz}
• Named output '{}' already alreadyDefined
• Named output '{namedOutput}' already alreadyDefined

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/4e19a92a192d6351. Report an issue: GitHub.