What causes "ConcatDeleteOp can only have {} sources at most." in apache/hadoop?

Split the work into batches of at most 1,048,576 sources: concat the first batch into the target, then concat each further batch into the same target

apache/hadoop · error · RuntimeException

ConcatDeleteOp can only have {} sources at most.

Error message

ConcatDeleteOp can only have {} sources at most.

What it means

ConcatDeleteOp.setSources() guards the source list of a concat edit record: MAX_CONCAT_SRC is 1024*1024 = 1,048,576 (FSEditLogOp.java:1250), and passing more sources throws this RuntimeException before the record is ever written to the edit log. It fires on the NameNode while building the edit op for a concat RPC, so the operation fails atomically rather than logging an unreadable record.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java:1275

      return cache.get(OP_CONCAT_DELETE);
    }

    @Override
    void resetSubFields() {
      length = 0;
      trg = null;
      srcs = null;
      timestamp = 0L;
    }

    ConcatDeleteOp setTarget(String trg) {
      this.trg = trg;
      return this;
    }

    ConcatDeleteOp setSources(String[] srcs) {
      if (srcs.length > MAX_CONCAT_SRC) {
        throw new RuntimeException("ConcatDeleteOp can only have " +
            MAX_CONCAT_SRC + " sources at most.");
      }
      this.srcs = srcs;

      return this;
    }

    ConcatDeleteOp setTimestamp(long timestamp) {
      this.timestamp = timestamp;
      return this;
    }

    @Override
    public void writeFields(DataOutputStream out) throws IOException {
      FSImageSerialization.writeString(trg, out);
            
      DeprecatedUTF8 info[] = new DeprecatedUTF8[srcs.length];
      int idx = 0;

View on GitHub (pinned to 2add963021)

Solutions

Split the work into batches of at most 1,048,576 sources: concat the first batch into the target, then concat each further batch into the same target
Validate srcs.length in the client before issuing the RPC and fail fast with a clear message
Check for a caller bug (accidental nesting/duplication) that inflated the source array

Example fix

// before
Path[] all = listAllFiles(srcDir);
fs.concat(trg, all);  // RuntimeException: ConcatDeleteOp can only have 1048576 sources at most.

// after
static final int MAX_CONCAT_SRC = 1024 * 1024;
for (int i = 0; i < all.length; i += MAX_CONCAT_SRC) {
  Path[] batch = Arrays.copyOfRange(all, i, Math.min(i + MAX_CONCAT_SRC, all.length));
  fs.concat(trg, batch);  // first batch merges into trg; later batches merge into the grown trg
}

Defensive patterns

Strategy: validation

Validate before calling

static final int MAX_CONCAT_SRC = 1024 * 1024; // must match FSEditLogOp.MAX_CONCAT_SRC
if (srcs == null || srcs.length == 0 || srcs.length > MAX_CONCAT_SRC) {
  throw new IllegalArgumentException(
      "concat sources must be in [1, " + MAX_CONCAT_SRC + "], got " + (srcs == null ? 0 : srcs.length));
}
fs.concat(trg, srcs);

Type guard

boolean isConcatSourceCountValid(int n) {
  return n >= 1 && n <= 1024 * 1024; // HDFS hard cap, FSEditLogOp.java:1250
}

Try / catch

try {
  fs.concat(trg, srcs);
} catch (RuntimeException e) {
  if (e.getMessage() != null && e.getMessage().contains("sources at most")) {
    // split into batches of <= 1048576 and retry
  } else { throw e; }
}

Prevention

Never pass an unbounded glob/listing result into concat; cap it in client code
Batch large merges at 1,048,576 sources or fewer per call
Fail fast with a clear client-side message instead of relying on the NameNode RuntimeException

When it happens

Trigger: A client calls DistributedFileSystem.concat(target, srcs) / ClientProtocol.concat with srcs.length > 1,048,576; the NameNode constructs the ConcatDeleteOp via setSources() and throws. In practice only generated code or a loop that expands a huge directory listing into one call produces this.

Common situations: Migration/compaction scripts that try to concatenate an entire directory tree in a single concat call; glob expansion feeding an unbounded array into concat.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/aea162e234832785. Report an issue: GitHub.