{"record":{"id":"6ac7e639820c57be","repo":"apache/hadoop","slug":"fail-to-get-block-checksum-for","errorCode":null,"errorMessage":"Fail to get block checksum for {}","messagePattern":"Fail to get block checksum for (.+?)","errorType":"exception","errorClass":"PathIOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/FileChecksumHelper.java","lineNumber":630,"sourceCode":"    }\n\n    @Override\n    void checksumBlocks() throws IOException {\n      int tmpTimeout = getClient().getConf().getChecksumEcSocketTimeout() * 1 +\n          getClient().getConf().getSocketTimeout();\n      setTimeout(tmpTimeout);\n\n      for (bgIdx = 0;\n           bgIdx < getLocatedBlocks().size() && getRemaining() >= 0; bgIdx++) {\n        if (isRefetchBlocks()) {  // refetch to get fresh tokens\n          refetchBlocks();\n        }\n\n        LocatedBlock locatedBlock = getLocatedBlocks().get(bgIdx);\n        LocatedStripedBlock blockGroup = (LocatedStripedBlock) locatedBlock;\n\n        if (!checksumBlockGroup(blockGroup)) {\n          throw new PathIOException(\n              getSrc(), \"Fail to get block checksum for \" + locatedBlock);\n        }\n      }\n    }\n\n\n    private boolean checksumBlockGroup(\n        LocatedStripedBlock blockGroup) throws IOException {\n      ExtendedBlock block = blockGroup.getBlock();\n      long requestedNumBytes = block.getNumBytes();\n      if (getRemaining() < block.getNumBytes()) {\n        requestedNumBytes = getRemaining();\n      }\n      setRemaining(getRemaining() - requestedNumBytes);\n\n      StripedBlockInfo stripedBlockInfo = new StripedBlockInfo(block,\n          blockGroup.getLocations(), blockGroup.getBlockTokens(),\n          blockGroup.getBlockIndices(), ecPolicy);","sourceCodeStart":612,"sourceCodeEnd":648,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/FileChecksumHelper.java#L612-L648","documentation":"The erasure-coded counterpart of the replicated block-checksum failure: StripedFileChecksumComputer iterates block groups, and if checksumBlockGroup() cannot obtain the group's checksum (reading data + parity chunks and reconstructing failed), it throws PathIOException with the file path and LocatedBlock. Because EC reconstructs from parity, this fires when reconstruction attempts inside the group also failed — e.g., more unavailable chunks than parity, or DataNode read errors on the chunks contacted.","triggerScenarios":"getFileChecksum() on an erasure-coded (e.g., RS-6-3) file when a block group's chunks cannot all be read or reconstructed: DataNodes for data chunks and parity chunks down simultaneously, corrupted chunks, or read timeouts during the striped read.","commonSituations":"EC files with multiple DataNode failures at or beyond the parity limit; slow/degraded DataNodes causing striped-read failures; verifying checksums of EC datasets during cluster maintenance windows.","solutions":["Check DataNode health (hdfs dfsadmin -report) and bring failed nodes back — EC tolerates up to parity-count failures","Run hdfs fsck /path -files -blocks -locations -replicaDetails to see damaged EC stripes","Retry when the cluster is stable; transient chunk-read failures are common during heavy load","If stripes exceed the failure threshold, restore data from source or re-distcp; also consider a lower-width EC policy for higher fault tolerance on critical data"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// Before checksumming EC files, confirm enough DataNodes are live:\n// parity tolerance = policy parity count; check with:\n//   hdfs dfsadmin -report | grep -c Live\n// and fsck the file: hdfs fsck /path -files -blocks -locations","typeGuard":null,"tryCatchPattern":"try {\n  return dfsClient.getFileChecksum(ecPath);\n} catch (PathIOException e) {\n  if (String.valueOf(e.getMessage()).contains(\"Fail to get block checksum\")) {\n    // striped reconstruction failed; wait for DataNode recovery, then retry\n    scheduleRetryWithBackoff(ecPath);\n  } else throw e;\n}","preventionTips":["Keep concurrent EC DataNode failures below the policy's parity count","Pause checksum/verification jobs during maintenance windows that restart multiple DataNodes","For critical EC datasets consider higher-parity policies (e.g., RS-3-2/RS-10-4) to widen the failure tolerance"],"tags":["hdfs","erasure-coding","checksum","block-group","datanode"],"backgroundTag":"all-replicas-unavailable","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}