apache/hadoop · error · IllegalArgumentException

replication must be >= 1

Error message

replication must be >= 1

What it means

SetReplication.processOptions (SetReplication.java:70) parses the first remaining argument as a short and throws IllegalArgumentException('replication must be >= 1') when the parsed value is below 1. A separate NumberFormatException path handles non-numeric input ('Illegal replication, a positive integer expected'), so this specific throw means the value WAS a valid short integer but was 0 or negative.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/shell/SetReplication.java:70

  protected short newRep = 0;
  protected List<PathData> waitList = new LinkedList<PathData>();
  protected boolean waitOpt = false;
  
  @Override
  protected void processOptions(LinkedList<String> args) throws IOException {
    CommandFormat cf = new CommandFormat(2, Integer.MAX_VALUE, "R", "w");
    cf.parse(args);
    waitOpt = cf.getOpt("w");
    setRecursive(true);
    
    try {
      newRep = Short.parseShort(args.removeFirst());
    } catch (NumberFormatException nfe) {
      displayWarning("Illegal replication, a positive integer expected");
      throw nfe;
    }
    if (newRep < 1) {
      throw new IllegalArgumentException("replication must be >= 1");
    }
  }

  @Override
  protected void processArguments(LinkedList<PathData> args)
  throws IOException {
    super.processArguments(args);
    if (waitOpt) waitForReplication();
  }

  @Override
  protected void processPath(PathData item) throws IOException {
    if (item.stat.isSymlink()) {
      throw new PathIOException(item.toString(), "Symlinks unsupported");
    }
    
    if (item.stat.isFile()) {
      // Do the checking if the file is erasure coded since

View on GitHub (pinned to 2add963021)

Solutions

1. Use a replication factor of at least 1: 'hadoop fs -setrep 3 /path'
2. Default the variable in scripts: REP=${REP:-3} before invoking -setrep
3. If the goal was cheaper storage, use erasure coding (HDFS EC policies) instead of replication 0 — EC files are skipped by setrep, not assigned 0

Example fix

# before
hadoop fs -setrep 0 /data/archive
# setrep: replication must be >= 1

# after
hadoop fs -setrep 1 /data/archive

short rep = Short.parseShort(repArg);
if (rep < 1) throw new IllegalArgumentException("replication must be >= 1");
// guard in scripts: REP=${REP:-3}; (( REP >= 1 )) || exit 1

catch (IllegalArgumentException e) when 'replication must be >= 1' -> clamp to a valid factor (>=1) or abort with a clear config error

When it happens

Trigger: 'hadoop fs -setrep 0 /path', 'hadoop fs -setrep -3 /path' (any integer < 1 as the first positional argument). Note: '-3' may be consumed as an option-looking token by CommandFormat first, but explicit 0 reliably reaches the check.

Common situations: Scripts computing replication from a variable that can be 0/unset; attempts to 'disable' replication by setting 0 (erasure-coded or archive intent); confusing replication factor with block size or memory settings in automation.

Related errors

• Could not set replication for:
• Invalid read parameters: buf.length=%d, off=%d, len=%d
• Move destination must be different from source for %s.
• Unsupported block buffer "{}"
• Cannot seek to a negative offset

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/0576f5604bd8a7c7. Report an issue: GitHub.