{"record":{"id":"7367a9979346fe0e","repo":"apache/hadoop","slug":"fetchblockbyterange-got-a-checksum-exception-fo","errorCode":null,"errorMessage":"fetchBlockByteRange(). Got a checksum exception for {} at {}:{} from {}","messagePattern":"fetchBlockByteRange\\(\\)\\. Got a checksum exception for (.+?) at (.+?):(.+?) from (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java","lineNumber":1260,"sourceCode":"\n        IOUtilsClient.updateReadStatistics(readStatistics, nread, reader);\n        dfsClient.updateFileSystemReadStats(\n            reader.getNetworkDistance(), nread, readTimeMS);\n        if (nread != len) {\n          throw new IOException(\"truncated return from reader.read(): \" +\n              \"excpected \" + len + \", got \" + nread);\n        }\n        DFSClientFaultInjector.get().readFromDatanodeDelay();\n        return;\n      } catch (ChecksumException e) {\n        String msg = \"fetchBlockByteRange(). Got a checksum exception for \"\n            + src + \" at \" + block.getBlock() + \":\" + e.getPos() + \" from \"\n            + datanode.info;\n        DFSClient.LOG.warn(msg);\n        // we want to remember what we have tried\n        corruptedBlocks.addCorruptedBlock(block.getBlock(), datanode.info);\n        addToLocalDeadNodes(datanode.info);\n        throw new IOException(msg);\n      } catch (IOException e) {\n        checkInterrupted(e);\n        if (e instanceof InvalidEncryptionKeyException && refetchEncryptionKey > 0) {\n          DFSClient.LOG.info(\"Will fetch a new encryption key and retry, \"\n              + \"encryption key was invalid when connecting to \" + datanode.addr\n              + \" : \" + e);\n          // The encryption key used is invalid.\n          refetchEncryptionKey--;\n          dfsClient.clearDataEncryptionKey();\n        } else if (refetchToken > 0 && tokenRefetchNeeded(e, datanode.addr)) {\n          refetchToken--;\n          try {\n            fetchBlockAt(block.getStartOffset());\n          } catch (IOException fbae) {\n            // ignore IOE, since we can retry it later in a loop\n          }\n        } else {\n          String msg = String.format(\"Failed to read block %s for file %s from datanode %s. \"","sourceCodeStart":1242,"sourceCodeEnd":1278,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java#L1242-L1278","documentation":"The DataNode raised a ChecksumException (CRC mismatch) for the requested range. The client logs it, records block+DN in corruptedBlocks (later reported to the NameNode as a bad block), adds the DN to the local dead list, and rethrows the message as a plain IOException so the outer loop retries a different replica. Persistent occurrences on all replicas indicate real on-disk corruption.","triggerScenarios":"CRC of the bytes served by the DN does not match the block's checksum metadata - bit rot, truncated checksum file, or wrong-version replica after crash recovery; read retried on another replica automatically.","commonSituations":"Failing disks producing silent corruption; corruption introduced during unsafe cluster restarts; all replicas corrupt after long undetected disk faults (then reads fail permanently).","solutions":["Let the client fail over to healthy replicas (it also reports the bad block to the NN)","Run hdfs fsck -files -blocks -locations to list corrupt replicas","Remove corrupt replicas with hdfs fsck -delete or -move so the NN re-replicates from a good copy","If every replica is corrupt, restore the file from source/backup and replace failing disks"],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":"static boolean isChecksumFailure(IOException e) {\n  return e.getMessage() != null && e.getMessage().contains(\"Got a checksum exception\");\n}","tryCatchPattern":"try {\n  in.readFully(buf, off, len);\n} catch (IOException e) {\n  if (isChecksumFailure(e)) {\n    // client retried other replicas and reported the bad block; surface a data-quality alert\n    alertCorruptBlock(e);\n  }\n  throw e;\n}","preventionTips":["Run scheduled hdfs fsck -files -blocks -locations to catch corrupt replicas proactively","Remove corrupt replicas (fsck -delete/-move) so good copies re-replicate","Monitor and replace disks that produce checksum errors"],"tags":["hdfs","checksum","corruption","datanode"],"backgroundTag":"checksum-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}