apache/hadoop · error · IOException

Block IDs mismatched: block${i}=${blocki}, expected block gr

Error message

Block IDs mismatched: block${i}=${blocki}, expected block group=${blockGroup}

What it means

Second invariant in StripedBlockUtil.checkBlocks(): for internal block i of a striped group, blockId(i) must equal groupBlockId + i — striped internal blocks have consecutive ids with the group id as base (the low 4 bits of a group id are the index mask, see getBlockIndex). If blocki.getBlockId() - i != blockGroup.getBlockId(), IOException is thrown during the striped write (DFSStripedOutputStream.waitEndBlocks).

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/StripedBlockUtil.java:1014

    @Override
    public String toString() {
      return String.format("StripeRange(offsetInBlock=%d, length=%d)",
          offsetInBlock, length);
    }
  }

  /**
   * Check if the information such as IDs and generation stamps in block-i
   * match the block group.
   */
  public static void checkBlocks(ExtendedBlock blockGroup,
      int i, ExtendedBlock blocki) throws IOException {
    if (!blocki.getBlockPoolId().equals(blockGroup.getBlockPoolId())) {
      throw new IOException("Block pool IDs mismatched: block" + i + "="
          + blocki + ", expected block group=" + blockGroup);
    }
    if (blocki.getBlockId() - i != blockGroup.getBlockId()) {
      throw new IOException("Block IDs mismatched: block" + i + "="
          + blocki + ", expected block group=" + blockGroup);
    }
    if (blocki.getGenerationStamp() != blockGroup.getGenerationStamp()) {
      throw new IOException("Generation stamps mismatched: block" + i + "="
          + blocki + ", expected block group=" + blockGroup);
    }
  }

  public static int getBlockIndex(Block reportedBlock) {
    long BLOCK_GROUP_INDEX_MASK = 15;
    return (int) (reportedBlock.getBlockId() &
        BLOCK_GROUP_INDEX_MASK);
  }
}

View on GitHub (pinned to 2add963021)

Solutions

1. Retry the write; single occurrences usually indicate a transient allocation race
2. Confirm uniform Hadoop versions across the cluster (block id layout is version-dependent)
3. Extract the block ids from the message and inspect them via `hdfs fsck` / block reports to confirm the group-id/index relationship is broken
4. Persistent recurrence: gather NN/DN logs for the block group id and file a Hadoop JIRA

try {
  try (FSDataOutputStream out = fs.create(ecPath)) { /* write striped data */ }
} catch (IOException e) {
  if (e.getMessage() != null && e.getMessage().contains("Block IDs mismatched")) {
    LOG.warn("Striped write failed on block id mismatch; retry job and check cluster versions", e);
  } else { throw e; }
}

When it happens

Trigger: Writing an EC file when a streamer's allocated block id is not contiguous with the block group id — i.e., the block returned by the DN/NN does not line up with the streamer's index in the group. Internal invariant violation, not a client-configurable condition.

Common situations: Version skew between NameNode and DataNodes on striped block id layout, or a regression in striped block allocation; manifests as EC write failures with 'Block IDs mismatched' in the client trace.

Related errors

• Block pool IDs mismatched: block${i}=${blocki}, expected blo
• Generation stamps mismatched: block${i}=${blocki}, expected
• All negative block group IDs are used, growing into positive
• FileSystem ${item.fs.getUri()} does not support Erasure Codi
• All positive block IDs are used, wrapping to negative IDs, w

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/035da95ae0185f0c. Report an issue: GitHub.