{"record":{"id":"ffb2912b21e0456f","repo":"apache/hadoop","slug":"checksum-error-at","errorCode":null,"errorMessage":"Checksum error: {} at {}","messagePattern":"Checksum error: (.+?) at (.+?)","errorType":"exception","errorClass":"ChecksumException","httpStatus":null,"severity":"critical","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java","lineNumber":316,"sourceCode":"          throw new ChecksumException(\n            \"Checksum file not a length multiple of checksum size \" +\n            \"in \" + file + \" at \" + pos + \" checksumpos: \" + checksumPos +\n            \" sumLenread: \" + sumLenRead,\n            pos);\n        }\n        if (sumLenRead <= 0) { // we're at the end of the file\n          eof = true;\n        } else {\n          // Adjust amount of data to read based on how many checksum chunks we read\n          len = Math.min(len, bytesPerSum * (sumLenRead / CHECKSUM_SIZE));\n        }\n      }\n      if(pos != datas.getPos()) {\n        datas.seek(pos);\n      }\n      int nread = readFully(datas, buf, offset, len);\n      if (eof && nread > 0) {\n        throw new ChecksumException(\"Checksum error: \"+file+\" at \"+pos, pos);\n      }\n      return nread;\n    }\n\n    /**\n     * Get the IO Statistics of the nested stream, falling back to\n     * null if the stream does not implement the interface\n     * {@link IOStatisticsSource}.\n     * @return an IOStatistics instance or null\n     */\n    @Override\n    public IOStatistics getIOStatistics() {\n      return IOStatisticsSupport.retrieveIOStatistics(datas);\n    }\n\n    public static long findChecksumOffset(long dataOffset,\n                                          int bytesPerSum) {\n      return HEADER_LENGTH + (dataOffset/bytesPerSum) * FSInputChecker.CHECKSUM_SIZE;","sourceCodeStart":298,"sourceCodeEnd":334,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L298-L334","documentation":"In ChecksumFileSystem's read path, `eof` is set when the .crc stream yields no more checksum records; if data bytes were still successfully read from the data file at that position (nread > 0), it throws ChecksumException \"Checksum error: <file> at <pos>\". Concretely: the data file is longer than its checksum coverage - the .<name>.crc is truncated, stale (file was appended after the crc was written), or belongs to a different version of the file.","triggerScenarios":"Reading past the original end of a locally-written file that was later appended by a process that did not update the .crc (plain java/OS appends, text editors); a truncated .crc that happened to end on a 4-byte boundary (so error 496 does not fire but coverage ends early); mixing files whose crc was written with different bytesPerChecksum.","commonSituations":"Log or output files on local staging are appended by non-Hadoop tooling while Hadoop clients later read them; out-of-band file modification between the Hadoop write and the read.","solutions":["Delete the stale .<name>.crc so the file is read without checksum coverage, or regenerate it by recopying the file through LocalFileSystem.","If the data must be trustworthy, restore both file and .crc from the authoritative source instead of bypassing.","For an immediate read, disable verification (fs.setVerifyChecksum(false) / `hadoop fs -cat -ignoreCrc`).","Fix the writer: only write/append local files through LocalFileSystem APIs so the .crc stays consistent."],"exampleFix":"// before\ntry (FSDataInputStream in = localFs.open(f)) { IOUtils.copyBytes(in, out, 4096); }\n\n// after: tolerate a stale .crc for this read\nlocalFs.setVerifyChecksum(false);\ntry (FSDataInputStream in = localFs.open(f)) { IOUtils.copyBytes(in, out, 4096); }\nlocalFs.setVerifyChecksum(true);","handlingStrategy":"try-catch","validationCode":"long dataLen = localFs.getFileStatus(f).getLen();\nlong covered = ((localFs.getFileStatus(crcOf(f)).getLen() - 8) / 4L) * bytesPerChecksum;\nif (dataLen > covered) {\n  // data extends beyond checksum coverage: drop the stale crc or re-copy the file\n}","typeGuard":null,"tryCatchPattern":"try {\n  IOUtils.copyBytes(localFs.open(f), out, 4096, true);\n} catch (ChecksumException e) {\n  localFs.setVerifyChecksum(false); // accept unverified read for this file only\n  IOUtils.copyBytes(localFs.open(f), out, 4096, true);\n  localFs.setVerifyChecksum(true);\n}","preventionTips":["Modify local files only through LocalFileSystem APIs so .crc stays in sync with appends","Do not append to Hadoop-written files with external tools","Regenerate the .crc (delete it and re-copy through LocalFileSystem) after out-of-band changes"],"tags":["hadoop","filesystem","checksum","local-filesystem","data-corruption"],"backgroundTag":"checksum-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}