apache/hadoop · critical · IllegalStateException

All positive block IDs are used, wrapping to negative IDs, w

Error message

All positive block IDs are used, wrapping to negative IDs, which might conflict with erasure coded block groups.

What it means

SequentialBlockIdGenerator hands out positive sequential IDs for replicated blocks, skipping any ID already referenced by a stored block (isValidBlock checks blockManager.getStoredBlock and its BlockCollectionId). If the sequence wraps past Long.MAX_VALUE into negative values it throws IllegalStateException, because negative IDs are reserved for erasure-coded block groups. The generator value is journaled, so the state persists across NameNode restarts.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/blockmanagement/SequentialBlockIdGenerator.java:58

  private final BlockManager blockManager;

  SequentialBlockIdGenerator(BlockManager blockManagerRef) {
    super(LAST_RESERVED_BLOCK_ID);
    this.blockManager = blockManagerRef;
  }

  @Override // NumberGenerator
  public long nextValue() {
    Block b = new Block(super.nextValue());

    // There may be an occasional conflict with randomly generated
    // block IDs. Skip over the conflicts.
    while(isValidBlock(b)) {
      b.setBlockId(super.nextValue());
    }
    if (b.getBlockId() < 0) {
      throw new IllegalStateException("All positive block IDs are used, " +
          "wrapping to negative IDs, " +
          "which might conflict with erasure coded block groups.");
    }
    return b.getBlockId();
  }

  /**
   * Returns whether the given block is one pointed-to by a file.
   */
  private boolean isValidBlock(Block b) {
    BlockInfo bi = blockManager.getStoredBlock(b);
    return bi != null && bi.getBlockCollectionId() !=
        INodeId.INVALID_INODE_ID;
  }
}

View on GitHub (pinned to 2add963021)

Solutions

1. Restore the namespace from the last clean fsimage checkpoint to reset the corrupted generation counter
2. If genuinely exhausted, rebuild the namespace via distcp (export data, reformat, re-import) — the positive long range cannot be extended
3. Report the case upstream with the fsimage generation values for diagnosis

try {
  out = fs.create(path);
} catch (RemoteException re) {
  if (re.getClassName().endsWith("IllegalStateException")
      && String.valueOf(re.getMessage()).contains("block IDs are used")) {
    alertOps("Positive block ID space exhausted on NameNode"); // no client-side workaround; metadata rebuild required
  } else throw re;
}

When it happens

Trigger: Allocating more than ~2^63 - initial blocks (astronomically large), or a corrupted persisted block generation counter in the fsimage/edits that starts near Long.MAX_VALUE. Thrown inside nextValue() while the NameNode assigns a new block ID for a replicated file write.

Common situations: Theoretical for real clusters; occurs with hand-edited or corrupted generation counters, or fault-injection tests that force counter values.

Related errors

• All negative block group IDs are used, growing into positive
• ${name}: serial number map is full
• ${name}: serial number ${i} does not exist
• Unknown nameservice: {}
• Configuration has multiple addresses that match local node's

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/536b1e441a3fa6f8. Report an issue: GitHub.