apache/hadoop · error · HadoopIllegalArgumentException

ReplicationFactor option is set multiple times

Error message

ReplicationFactor option is set multiple times

What it means

AbstractFileSystem.create allows CreateOpts.ReplicationFactor at most once; a second instance in the varargs throws HadoopIllegalArgumentException during argument processing. The check exists so the effective replication value is never ambiguous.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java:560

    Progressable progress = null;
    Boolean createParent = null;
 
    for (CreateOpts iOpt : opts) {
      if (CreateOpts.BlockSize.class.isInstance(iOpt)) {
        if (blockSize != -1) {
          throw new HadoopIllegalArgumentException(
              "BlockSize option is set multiple times");
        }
        blockSize = ((CreateOpts.BlockSize) iOpt).getValue();
      } else if (CreateOpts.BufferSize.class.isInstance(iOpt)) {
        if (bufferSize != -1) {
          throw new HadoopIllegalArgumentException(
              "BufferSize option is set multiple times");
        }
        bufferSize = ((CreateOpts.BufferSize) iOpt).getValue();
      } else if (CreateOpts.ReplicationFactor.class.isInstance(iOpt)) {
        if (replication != -1) {
          throw new HadoopIllegalArgumentException(
              "ReplicationFactor option is set multiple times");
        }
        replication = ((CreateOpts.ReplicationFactor) iOpt).getValue();
      } else if (CreateOpts.BytesPerChecksum.class.isInstance(iOpt)) {
        if (bytesPerChecksum != -1) {
          throw new HadoopIllegalArgumentException(
              "BytesPerChecksum option is set multiple times");
        }
        bytesPerChecksum = ((CreateOpts.BytesPerChecksum) iOpt).getValue();
      } else if (CreateOpts.ChecksumParam.class.isInstance(iOpt)) {
        if (checksumOpt != null) {
          throw new  HadoopIllegalArgumentException(
              "CreateChecksumType option is set multiple times");
        }
        checksumOpt = ((CreateOpts.ChecksumParam) iOpt).getValue();
      } else if (CreateOpts.Perms.class.isInstance(iOpt)) {
        if (permission != null) {
          throw new HadoopIllegalArgumentException(

View on GitHub (pinned to 2add963021)

Solutions

1. Ensure replication is passed once; delete the redundant CreateOpts.replication(...) from one of the layers
2. Use CreateOpts.setOpt(CreateOpts.replication(r), opts) to replace any existing value
3. Add a dedupe-by-class step in shared create wrappers

Example fix

// before
fc.create(f, flag, CreateOpts.replication((short) 3),
    CreateOpts.replication((short) 2));

// after
fc.create(f, flag, CreateOpts.replication((short) 3));

opts = CreateOpts.setOpt(CreateOpts.replication(r), opts); // replaces any existing replication option

When it happens

Trigger: Two CreateOpts.replication(...) entries in one create() call — commonly a caller-supplied replication plus a library-injected default, or code merging per-job config opts with per-call opts by concatenation.

Common situations: Write helpers that enforce a default replication; pipelines that layer options (caller -> service -> client shim) where each layer appends instead of replacing; duplicated code paths both adding the option.

Related errors

• BlockSize option is set multiple times
• BufferSize option is set multiple times
• BytesPerChecksum option is set multiple times
• CreateChecksumType option is set multiple times
• Perms option is set multiple times

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/566ee27147cac799. Report an issue: GitHub.