apache/hadoop · critical · IOException

invalid negative number of blocks

Error message

invalid negative number of blocks

What it means

readBlocks reads the block count of an OP_ADD or OP_CLOSE record; a negative count cannot occur in valid data, so it throws IOException('invalid negative number of blocks'). In practice the stream position is wrong: earlier corruption desynchronized the reader and the count field now holds arbitrary bits.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java:694

          this.erasureCodingPolicyId =
              ErasureCodeConstants.REPLICATION_POLICY_ID;
        }
        // read clientId and callId
        readRpcIds(in, logVersion);
      } else {
        this.clientName = "";
        this.clientMachine = "";
      }
    }

    static final public int MAX_BLOCKS = 1024 * 1024 * 64;
    
    private static Block[] readBlocks(
        DataInputStream in,
        int logVersion) throws IOException {
      int numBlocks = in.readInt();
      if (numBlocks < 0) {
        throw new IOException("invalid negative number of blocks");
      } else if (numBlocks > MAX_BLOCKS) {
        throw new IOException("invalid number of blocks: " + numBlocks +
            ".  The maximum number of blocks per file is " + MAX_BLOCKS);
      }
      Block[] blocks = new Block[numBlocks];
      for (int i = 0; i < numBlocks; i++) {
        Block blk = new Block();
        blk.readFields(in);
        blocks[i] = blk;
      }
      return blocks;
    }

    public String stringifyMembers() {
      StringBuilder builder = new StringBuilder();
      builder.append("[length=")
          .append(length)
          .append(", inodeId=")

View on GitHub (pinned to 2add963021)

Solutions

1. Run 'hdfs namenode -recover' so the loader resyncs past the bad record
2. Validate the segment with 'hdfs oev' and restore fsimage plus edits from backup if it fails
3. Re-copy any segment that was transferred partially (compare sizes and checksums)

Example fix

# before: replay fails 'invalid negative number of blocks'
hdfs --daemon start namenode

# after: confirm the bad segment, then recover
hdfs oev -i <segment> -o /tmp/check.xml -p xml   # fails at the same offset
hdfs namenode -recover

hdfs oev -i <segment> -o /tmp/check.xml -p xml   # parse fails at the desynchronized record

try {
  loader.loadFSEdits(storage, 0);
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("invalid negative number of blocks")) {
    // reader desynchronized on corrupt bytes: 'hdfs namenode -recover'
    // resyncs past it, or restore fsimage+edits from backup
  }
  throw e;
}

When it happens

Trigger: Decoding a corrupt or truncated Add/Close record where garbage lands in the numBlocks field; reading a segment under wrong layout assumptions after desync.

Common situations: Torn final segment after a NameNode crash; disk corruption; partially copied segment files.

Related errors

• Mismatched block IDs or generation stamps for the old last b
• Mismatched block IDs or generation stamps, attempting to rep
• Trying to remove more than one block from file {}
• Trying to delete non-existant block {}
• Incorrect data format. logVersion is {} but writables.length

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/17aeb934732c4427. Report an issue: GitHub.