{"record":{"id":"7c0552fcdd573b1a","repo":"apache/hadoop","slug":"checksum-error-at-exp-got","errorCode":null,"errorMessage":"Checksum error: {} at {} exp: {} got: {}","messagePattern":"Checksum error: (.+?) at (.+?) exp: (.+?) got: (.+?)","errorType":"exception","errorClass":"CompletionException","httpStatus":null,"severity":"critical","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java","lineNumber":431,"sourceCode":"          // During last chunk, there may be less than chunk size\n          // data preset, so setting the limit accordingly.\n          int lastIncompleteChunk = data.remaining() % bytesPerSum;\n          current.limit((c * bytesPerSum) + lastIncompleteChunk);\n        } else {\n          // set the buffer limit to end of every chunk.\n          current.limit((c + 1) * bytesPerSum);\n        }\n\n        // compute the crc\n        crc.reset();\n        crc.update(current);\n        int expected = sums.get();\n        int calculated = (int) crc.getValue();\n\n        if (calculated != expected) {\n          // cast of c added to silence findbugs\n          long errPosn = dataOffset + (long) c * bytesPerSum;\n          throw new CompletionException(new ChecksumException(\n              \"Checksum error: \" + file + \" at \" + errPosn +\n                  \" exp: \" + expected + \" got: \" + calculated, errPosn));\n        }\n      }\n      // if everything matches, we return the data\n      return data;\n    }\n\n    /**\n     * Turn off range merging to make buffer recycling more likely (but not guaranteed).\n     * @return 0, always\n     */\n    @Override\n    public int maxReadSizeForVectorReads() {\n      return S_0;\n    }\n\n    /**","sourceCodeStart":413,"sourceCodeEnd":449,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/ChecksumFileSystem.java#L413-L449","documentation":"This is ChecksumFileSystem's ByteBuffer read path verifying data chunk by chunk: it CRC32s each bytesPerChecksum (default 512) slice of the returned buffer and compares against the expected value read from the .<name>.crc file. A mismatch throws CompletionException wrapping ChecksumException with the file, the exact failing offset, and both expected and computed CRCs. Detected here means the local data actually differs from what was written - real corruption (bad disk, bit rot, bad copy), not a Hadoop bug.","triggerScenarios":"Reading a local file through LocalFileSystem (or another ChecksumFileSystem) whose data block no longer matches its .crc: failing disk sectors, silent corruption on NFS/network mounts, a file copied without its crc and paired with a stale crc from another file, or partial writes from a crashed job.","commonSituations":"Jobs reading local staging/spill data after hardware errors; directories synced with tools that updated the data file but preserved an old .crc; long-lived local caches on unreliable disks.","solutions":["Treat it as data corruption: restore the file (and its .crc) from the authoritative source or backup - retrying the same read will keep failing.","Verify the media: check dmesk/smartctl for disk errors and re-copy the file, then confirm with md5sum against the source.","If integrity is already assured by other means and you accept the risk, bypass with fs.setVerifyChecksum(false) or fs.file.impl=RawLocalFileSystem.","Delete the .crc only if you have confirmed the data is good (e.g. md5 matches source); otherwise you are just hiding corruption."],"exampleFix":"// before\ntry (FSDataInputStream in = localFs.open(f)) {\n  ByteBuffer b = in.read(pool, 65536, EnumSet.noneOf(ReadOption.class));\n}\n\n// after: surface corruption and recover from source\ntry (FSDataInputStream in = localFs.open(f)) {\n  ByteBuffer b = in.read(pool, 65536, EnumSet.noneOf(ReadOption.class));\n} catch (CompletionException e) {\n  if (e.getCause() instanceof ChecksumException) { /* re-fetch file from source */ }\n  else throw e;\n}","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  ByteBuffer b = in.read(pool, 65536, EnumSet.noneOf(ReadOption.class));\n} catch (CompletionException ce) {\n  Throwable cause = ce.getCause();\n  if (cause instanceof ChecksumException) {\n    // real local corruption: re-fetch the file from the source, do not retry this stream\n    restoreFromSource(path);\n  } else {\n    throw ce;\n  }\n}","preventionTips":["Keep authoritative copies of important local staging data so corruption is recoverable","Monitor disk health (SMART) on nodes holding local checksummed data","Never delete a .crc to silence this error unless the data was verified against the source (md5)","Use `hadoop fs -cat -ignoreCrc` / setVerifyChecksum(false) only as a last-resort diagnostic"],"tags":["hadoop","filesystem","checksum","local-filesystem","data-corruption"],"backgroundTag":"checksum-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}