apache/hadoop · error · RuntimeException

Can't have more than {} in an AddCloseOp.

Error message

Can't have more than {} in an AddCloseOp.

What it means

AddCloseOp.setBlocks rejects block arrays longer than MAX_BLOCKS (1024*1024*64 = 67108864) with a RuntimeException. This is a writer-side sanity bound while constructing the edit op. A real file cannot reach that many blocks, so hitting it means programmatic misuse (building ops by hand) or corrupted upstream state.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java:511

    <T extends AddCloseOp> T setModificationTime(long mtime) {
      this.mtime = mtime;
      return (T)this;
    }

    <T extends AddCloseOp> T setAccessTime(long atime) {
      this.atime = atime;
      return (T)this;
    }

    <T extends AddCloseOp> T setBlockSize(long blockSize) {
      this.blockSize = blockSize;
      return (T)this;
    }

    <T extends AddCloseOp> T setBlocks(Block[] blocks) {
      if (blocks.length > MAX_BLOCKS) {
        throw new RuntimeException("Can't have more than " + MAX_BLOCKS +
            " in an AddCloseOp.");
      }
      this.blocks = FSEditLogOp.deepCopy(blocks);
      return (T)this;
    }
    
    @Override
    public Block[] getBlocks() {
      return blocks;
    }

    <T extends AddCloseOp> T setPermissionStatus(PermissionStatus permissions) {
      this.permissions = permissions;
      return (T)this;
    }

    <T extends AddCloseOp> T setAclEntries(List<AclEntry> aclEntries) {
      this.aclEntries = aclEntries;

View on GitHub (pinned to 2add963021)

Solutions

1. Check blocks.length against AddCloseOp.MAX_BLOCKS before calling setBlocks
2. If state claims that many blocks for one file, treat the state as corrupt and investigate the writer
3. Split or shrink synthetic test data so each op stays under the bound

Example fix

// before
op.setBlocks(file.getBlocks());   // RuntimeException if over MAX_BLOCKS

// after
Block[] blocks = file.getBlocks();
if (blocks.length > AddCloseOp.MAX_BLOCKS) {
  throw new IllegalArgumentException("File has " + blocks.length
      + " blocks; AddCloseOp supports at most " + AddCloseOp.MAX_BLOCKS);
}
op.setBlocks(blocks);

Block[] blocks = file.getBlocks();
Preconditions.checkArgument(blocks.length <= FSEditLogOp.AddCloseOp.MAX_BLOCKS,
    "File carries %s blocks; AddCloseOp supports at most %s",
    blocks.length, FSEditLogOp.AddCloseOp.MAX_BLOCKS);
op.setBlocks(blocks);

try {
  op.setBlocks(blocks);
} catch (RuntimeException e) {
  if (e.getMessage() != null && e.getMessage().contains("MAX_BLOCKS")) {
    // synthetic data exceeded the 64M-block bound: shrink the input,
    // the limit itself is correct
  }
  throw e;
}

When it happens

Trigger: Calling setBlocks on an AddCloseOp (or its subclasses in log-generation code and tests) with an array larger than 64Mi entries; synthesizing ops from state where a bogus block count produced an oversized array.

Common situations: Unit tests and tooling that construct synthetic edit ops; log fuzzers feeding random block counts.

Related errors

• "Cannot find a block policy with the name " + policyName
• Invalid class specified for {}
• No class configured for {}
• Unable to construct journal, {}
• Error replaying edit log at offset {}. Expected transaction

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/8f2153f90ca39f64. Report an issue: GitHub.