apache/hadoop · error · HadoopIllegalArgumentException

${flag}Both append and overwrite options cannot be enabled.

Error message

${flag}Both append and overwrite options cannot be enabled.

What it means

CreateFlag.validate refuses a set containing both APPEND and OVERWRITE with HadoopIllegalArgumentException (note the message concatenates the flag set straight into 'Both append and overwrite...' with no separator). The two options describe contradictory write modes - resume at end-of-file vs replace from byte zero - so the filesystem cannot honor both.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CreateFlag.java:159

    return mode;
  }
  
  /**
   * Validate the CreateFlag and throw exception if it is invalid
   * @param flag set of CreateFlag
   * @throws HadoopIllegalArgumentException if the CreateFlag is invalid
   */
  public static void validate(EnumSet<CreateFlag> flag) {
    if (flag == null || flag.isEmpty()) {
      throw new HadoopIllegalArgumentException(flag
          + " does not specify any options");
    }
    final boolean append = flag.contains(APPEND);
    final boolean overwrite = flag.contains(OVERWRITE);
    
    // Both append and overwrite is an error
    if (append && overwrite) {
      throw new HadoopIllegalArgumentException(
          flag + "Both append and overwrite options cannot be enabled.");
    }
  }
  
  /**
   * Validate the CreateFlag for create operation
   * @param path Object representing the path; usually String or {@link Path}
   * @param pathExists pass true if the path exists in the file system
   * @param flag set of CreateFlag
   * @throws IOException on error
   * @throws HadoopIllegalArgumentException if the CreateFlag is invalid
   */
  public static void validate(Object path, boolean pathExists,
      EnumSet<CreateFlag> flag) throws IOException {
    validate(flag);
    final boolean append = flag.contains(APPEND);
    final boolean overwrite = flag.contains(OVERWRITE);
    if (pathExists) {

View on GitHub (pinned to 2add963021)

Solutions

1. Pick one mode: EnumSet.of(CreateFlag.APPEND) to keep existing bytes, or EnumSet.of(CreateFlag.CREATE, CreateFlag.OVERWRITE) to replace the file.
2. If you merge flag sets, resolve the conflict explicitly first: if (merged.contains(APPEND)) merged.remove(OVERWRITE);.
3. Validate early with CreateFlag.validate(flags) so the error points at your own code, and treat both-set as a usage/configuration error upstream.

Example fix

// before
EnumSet<CreateFlag> flags = EnumSet.noneOf(CreateFlag.class);
if (cfg.append()) flags.add(CreateFlag.APPEND);
if (cfg.overwrite()) flags.add(CreateFlag.OVERWRITE); // both true -> throws
fs.create(path, perms, flags, 4096, (short)1, 1<<26, null);

// after: overwrite wins, append only when not overwriting
EnumSet<CreateFlag> flags = cfg.overwrite()
    ? EnumSet.of(CreateFlag.CREATE, CreateFlag.OVERWRITE)
    : (cfg.append() ? EnumSet.of(CreateFlag.APPEND)
                     : EnumSet.of(CreateFlag.CREATE));

if (flags.contains(CreateFlag.APPEND)) flags.remove(CreateFlag.OVERWRITE);
CreateFlag.validate(flags);

static boolean validFlagCombo(EnumSet<CreateFlag> f) {
  return !f.isEmpty() && !(f.contains(CreateFlag.APPEND) && f.contains(CreateFlag.OVERWRITE));
}

try {
  fs.create(path, perms, flags, buf, rep, block, null);
} catch (HadoopIllegalArgumentException e) { // 'Both append and overwrite...'
  // resolve to a single mode: keep APPEND, drop OVERWRITE (or vice versa), retry
}

When it happens

Trigger: Passing EnumSet.of(CreateFlag.APPEND, CreateFlag.OVERWRITE) (directly or via a builder/union of two option groups, e.g., flags = EnumSet.of(APPEND); flags.addAll(otherFlags)) to fs.create(...)/fc.create(...) or CreateFlag.validate.

Common situations: Combining CLI-style booleans into flags ('-append' and '-overwrite' both given to a tool like distcp/ FsShell wrapper); merging a default flag set with a user-supplied one without removing the conflicting member; copy-pasted flag literals.

Related errors

• ${flag} does not specify any options
• ${flag} does not contain APPEND
• relative paths not allowed:{path}
• The source {src} and destination {dst} are the same
• File already exists: ${path}. Append or overwrite option mus

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/76ce786e01759a11. Report an issue: GitHub.