{"record":{"id":"ba5215ad02c7cb89","repo":"apache/hadoop","slug":"generation-stamps-mismatched-block-i-blocki","errorCode":null,"errorMessage":"Generation stamps mismatched: block${i}=${blocki}, expected block group=${blockGroup}","messagePattern":"Generation stamps mismatched: block(.+?)=(.+?), expected block group=(.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/StripedBlockUtil.java","lineNumber":1018,"sourceCode":"    }\n  }\n\n  /**\n   * Check if the information such as IDs and generation stamps in block-i\n   * match the block group.\n   */\n  public static void checkBlocks(ExtendedBlock blockGroup,\n      int i, ExtendedBlock blocki) throws IOException {\n    if (!blocki.getBlockPoolId().equals(blockGroup.getBlockPoolId())) {\n      throw new IOException(\"Block pool IDs mismatched: block\" + i + \"=\"\n          + blocki + \", expected block group=\" + blockGroup);\n    }\n    if (blocki.getBlockId() - i != blockGroup.getBlockId()) {\n      throw new IOException(\"Block IDs mismatched: block\" + i + \"=\"\n          + blocki + \", expected block group=\" + blockGroup);\n    }\n    if (blocki.getGenerationStamp() != blockGroup.getGenerationStamp()) {\n      throw new IOException(\"Generation stamps mismatched: block\" + i + \"=\"\n          + blocki + \", expected block group=\" + blockGroup);\n    }\n  }\n\n  public static int getBlockIndex(Block reportedBlock) {\n    long BLOCK_GROUP_INDEX_MASK = 15;\n    return (int) (reportedBlock.getBlockId() &\n        BLOCK_GROUP_INDEX_MASK);\n  }\n}\n","sourceCodeStart":1000,"sourceCodeEnd":1029,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/util/StripedBlockUtil.java#L1000-L1029","documentation":"Third invariant in StripedBlockUtil.checkBlocks(): every internal block i of a striped group must carry the same generation stamp as the group. blocki.getGenerationStamp() != blockGroup.getGenerationStamp() throws IOException from DFSStripedOutputStream.waitEndBlocks, aborting the erasure-coded write. Generation stamps are bumped during block recovery, so a mismatch means one internal block was recovered independently of the group view.","triggerScenarios":"Writing an EC file when a datanode failure triggers pipeline recovery that rolls an internal block's generation stamp while the writer still compares against the original group stamp — recovery did not propagate the new stamp to the whole group view.","commonSituations":"Struggles during rolling upgrades or datanode restarts mid-write of striped files; historically associated with striped-pipeline recovery bugs. Users see the EC write fail after a DN drop.","solutions":["Retry the write — after recovery completes, a fresh block group gets consistent stamps","Check DataNode health and network stability; repeated mid-write DN failures keep re-triggering recovery","Verify uniform Hadoop versions (generation stamp propagation for striped groups changed across releases)","If a single DN's failure reliably reproduces it, capture its logs plus the stamps from the message and report upstream"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  try (FSDataOutputStream out = fs.create(ecPath)) { /* write striped data */ }\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"Generation stamps mismatched\")) {\n    // striped recovery left inconsistent stamps; retry gets a fresh block group\n    LOG.warn(\"Striped write failed on generation stamp mismatch, retrying\", e);\n  } else { throw e; }\n}","preventionTips":["Keep DataNodes stable during large EC writes to avoid mid-write recovery","Monitor DN restart/decommission events when striped writes fail repeatedly"],"tags":["hdfs","erasure-coding","striped-blocks","generation-stamp","write-path"],"backgroundTag":"block-metadata-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}