What causes "${flag} does not contain APPEND" in apache/hadoop?

Pass EnumSet.of(CreateFlag.APPEND) on the append path (add CREATE as well if you want create-if-missing where supported).

apache/hadoop · error · HadoopIllegalArgumentException

${flag} does not contain APPEND

Error message

${flag} does not contain APPEND

What it means

CreateFlag.validateForAppend (called by FileSystem.append and FileContext append paths before opening) first runs the generic validate, then requires APPEND to be present, throwing HadoopIllegalArgumentException when it is not. Its purpose is to reject flag sets that came through the append API without actually expressing append semantics.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/CreateFlag.java:198

            + path.toString()
            + ". Append or overwrite option must be specified in " + flag);
      }
    } else if (!flag.contains(CREATE)) {
      throw new FileNotFoundException("Non existing file: " + path.toString()
          + ". Create option is not specified in " + flag);
    }
  }

  /**
   * Validate the CreateFlag for the append operation. The flag must contain
   * APPEND, and cannot contain OVERWRITE.
   *
   * @param flag enum set flag.
   */
  public static void validateForAppend(EnumSet<CreateFlag> flag) {
    validate(flag);
    if (!flag.contains(APPEND)) {
      throw new HadoopIllegalArgumentException(flag
          + " does not contain APPEND");
    }
  }
}

View on GitHub (pinned to 2add963021)

Solutions

Pass EnumSet.of(CreateFlag.APPEND) on the append path (add CREATE as well if you want create-if-missing where supported).
Route by intent in the helper: if (flags.contains(APPEND)) fs.append(...) else fs.create(...) instead of always calling append.
Unit-test both flag shapes through your helper so an invalid combination fails in CI, not in production.

Example fix

// before
public FSDataOutputStream openForWrite(Path p, EnumSet<CreateFlag> flags) {
  return fs.append(p, 4096); // internally validates: flags without APPEND -> throws
}

// after
public FSDataOutputStream openForWrite(Path p, EnumSet<CreateFlag> flags) {
  return flags.contains(CreateFlag.APPEND)
      ? fs.append(p, 4096)
      : fs.create(p, flags.contains(CreateFlag.OVERWRITE));
}

Defensive patterns

Strategy: validation

Validate before calling

if (!flags.contains(CreateFlag.APPEND)) {
  // route to create() instead of append(); CreateFlag.validateForAppend(flags) would throw
}

Type guard

static boolean isAppendCall(EnumSet<CreateFlag> f) {
  return f.contains(CreateFlag.APPEND);
}

Try / catch

try {
  CreateFlag.validateForAppend(flags);
} catch (HadoopIllegalArgumentException e) { // '... does not contain APPEND'
  // correct the flag set before calling fs.append
}

Prevention

Branch on flags.contains(APPEND) before choosing the fs.append vs fs.create call.
Keep append and create flag literals in one constants class so they cannot be mixed up.

When it happens

Trigger: Calling fs.append(f, bufferSize, progress) / fc.append where the implementation funnels into CreateFlag.validateForAppend with a set lacking APPEND - typically hand-built sets like EnumSet.of(CreateFlag.CREATE) passed into a generic 'open for write' helper that ends up on the append branch, or direct misuse of validateForAppend in tests.

Common situations: Unified open-for-write helpers that pick append vs create from flags but always delegate to fs.append(); test code exercising flag validation; copy-pasted flag sets from create paths reused in append paths.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/bfd52d9106f90a14. Report an issue: GitHub.