{"record":{"id":"06f85b538a1f34b0","repo":"apache/hadoop","slug":"expected-checksum-is-s-while-actual-checksum-is-06f85b","errorCode":null,"errorMessage":"Expected checksum is %s while actual checksum is %s","messagePattern":"Expected checksum is (.+?) while actual checksum is (.+?)","errorType":"exception","errorClass":"ChecksumMismatchException","httpStatus":null,"severity":"critical","filePath":"hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/object/tos/GetObjectOutput.java","lineNumber":51,"sourceCode":"\n  public GetObjectOutput(GetObjectV2Output output, byte[] checksum) {\n    Preconditions.checkNotNull(checksum, \"Checksum should not be null.\");\n    this.output = output;\n    this.checksum = checksum;\n  }\n\n  public GetObjectV2Output output() {\n    return output;\n  }\n\n  public byte[] checksum() {\n    return checksum;\n  }\n\n  public InputStream verifiedContent(byte[] expectedChecksum) throws IOException {\n    if (!Arrays.equals(expectedChecksum, checksum)) {\n      CommonUtils.runQuietly(this::forceClose);\n      throw new ChecksumMismatchException(expectedChecksum, checksum);\n    }\n\n    return output.getContent();\n  }\n\n  public void forceClose() throws IOException {\n    output.forceClose();\n  }\n}\n","sourceCodeStart":33,"sourceCodeEnd":61,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-cloud-storage-project/hadoop-tos/src/main/java/org/apache/hadoop/fs/tosfs/object/tos/GetObjectOutput.java#L33-L61","documentation":"GetObjectOutput wraps a GetObjectV2Output plus the checksum parsed from the TOS response headers (crc64ecma/crc32c). verifiedContent(expected) compares the two and, on mismatch, force-closes the connection (freeing the socket) and throws ChecksumMismatchException (\"Expected checksum is %s while actual checksum is %s\"), an IOException. It guards range/random reads against corruption or stale expectations.","triggerScenarios":"Range-read verification where the expected checksum was captured from a different object state: object overwritten between head and GET, expected computed under a different fs.tos.checksum-type (CRC32C vs CRC64ECMA), or the payload corrupted in transit.","commonSituations":"Concurrent writers replacing the same key; mixed client configurations with different checksum types; flaky proxies altering partial response bodies.","solutions":["Re-fetch objectStatus(key) for a fresh expected checksum and retry the read once","Pin fs.tos.checksum-type to one value (default CRC64ECMA) across the cluster","Enforce single-writer per key or version your expectations","If the mismatch is repeatable on stable data, treat it as corruption and restore the object"],"exampleFix":"// before\ntry (InputStream in = getOutput.verifiedContent(expected)) {\n  consume(in);\n}\n\n// after\ntry (InputStream in = getOutput.verifiedContent(expected)) {\n  consume(in);\n} catch (ChecksumMismatchException e) {\n  byte[] fresh = storage.objectStatus(key).checksum();\n  try (ObjectContent c = storage.get(key, offset, len);\n       InputStream in = c.verifiedStream(fresh)) {\n    consume(in);\n  }\n}","handlingStrategy":"retry","validationCode":"byte[] expected = storage.objectStatus(key).checksum();\ntry (InputStream in = getOutput.verifiedContent(expected)) {\n  consume(in);\n}","typeGuard":null,"tryCatchPattern":"try (InputStream in = getOutput.verifiedContent(expected)) {\n  consume(in);\n} catch (ChecksumMismatchException e) {\n  byte[] fresh = storage.objectStatus(key).checksum();\n  try (ObjectContent c = storage.get(key, offset, length);\n       InputStream in = c.verifiedStream(fresh)) {\n    consume(in);\n  }\n}","preventionTips":["Fetch expected checksums from live metadata, not cached values","Pin fs.tos.checksum-type cluster-wide","Enforce single-writer per key to avoid mid-read overwrites"],"tags":["hadoop-tos","volcengine-tos","data-integrity","checksum","range-read"],"backgroundTag":"checksum-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}