apache/hadoop · error · HadoopIllegalArgumentException

SHOULD_REPLICATE flag and ecPolicyName are exclusive paramet

Error message

SHOULD_REPLICATE flag and ecPolicyName are exclusive parameters. Set both is not allowed!

What it means

A single create call cannot both force replicated layout (CreateFlag.SHOULD_REPLICATE) and name an erasure-coding policy via ecPolicyName — the file layout must come from exactly one source. startFile detects the combination before locking and throws HadoopIllegalArgumentException.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSNamesystem.java:2813

          .append(", blockSize=").append(blockSize)
          .append(", supportedVersions=")
          .append(Arrays.toString(supportedVersions))
          .append(", ecPolicyName=").append(ecPolicyName)
          .append(", storagePolicy=").append(storagePolicy);
      NameNode.stateChangeLog.debug(builder.toString());
    }
    if (!DFSUtil.isValidName(src) ||
        FSDirectory.isExactReservedName(src) ||
        (FSDirectory.isReservedName(src)
            && !FSDirectory.isReservedRawName(src)
            && !FSDirectory.isReservedInodesName(src))) {
      throw new InvalidPathException(src);
    }

    boolean shouldReplicate = flag.contains(CreateFlag.SHOULD_REPLICATE);
    if (shouldReplicate &&
        (!org.apache.commons.lang3.StringUtils.isEmpty(ecPolicyName))) {
      throw new HadoopIllegalArgumentException("SHOULD_REPLICATE flag and " +
          "ecPolicyName are exclusive parameters. Set both is not allowed!");
    }

    INodesInPath iip = null;
    boolean skipSync = true; // until we do something that might create edits
    HdfsFileStatus stat = null;
    BlocksMapUpdateInfo toRemoveBlocks = null;

    checkOperation(OperationCategory.WRITE);
    final FSPermissionChecker pc = getPermissionChecker();
    writeLock(RwLockMode.FS);
    try {
      checkOperation(OperationCategory.WRITE);
      checkNameNodeSafeMode("Cannot create file" + src);

      iip = FSDirWriteFileOp.resolvePathForStartFile(
          dir, pc, src, flag, createParent);

View on GitHub (pinned to 2add963021)

Solutions

1. Drop CreateFlag.SHOULD_REPLICATE from the flag set and keep ecPolicyName (EC layout)
2. Or pass ecPolicyName null/empty and keep SHOULD_REPLICATE (replicated layout)

Example fix

// before
fs.create(path, FsPermission.getDefault(),
    EnumSet.of(CreateFlag.CREATE, CreateFlag.SHOULD_REPLICATE), 4096, (short) 3, 1 << 20, null, "RS-6-3-1024k");
// after: pick exactly one layout source
fs.create(path, FsPermission.getDefault(),
    EnumSet.of(CreateFlag.CREATE), 4096, (short) 3, 1 << 20, null, "RS-6-3-1024k");

boolean replicate = flags.contains(CreateFlag.SHOULD_REPLICATE);
boolean hasEc = ecPolicyName != null && !ecPolicyName.isEmpty();
if (replicate && hasEc) throw new IllegalArgumentException("pick SHOULD_REPLICATE or ecPolicyName, not both");

When it happens

Trigger: DFSClient.create with a flag set containing CreateFlag.SHOULD_REPLICATE and a non-empty ecPolicyName — e.g., wrapper code that sets SHOULD_REPLICATE whenever replication > 0 while also forwarding an EC policy parameter.

Common situations: Migration code that hardcodes the replicate flag for legacy layout and later adds the ecPolicyName parameter; frameworks whose create wrapper fills both fields; copy-paste from an EC example into code that already sets the flag.

Related errors

• FileSystem ${item.fs.getUri()} does not support Erasure Codi
• All negative block group IDs are used, growing into positive
• Unknown BlockChecksumType: " + groupChecksumType
• Byte-per-checksum not matched: bpc={} but bytesPerCRC={}
• BlockChecksumType COMPOSITE_CRC doesn't support MIXED underl

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/5f044d451b480b31. Report an issue: GitHub.