{"record":{"id":"0f0ef448c2a3817a","repo":"apache/druid","slug":"expected-footer-checksum-did-not-match-actual-chec","errorCode":null,"errorMessage":"Expected footer checksum did not match actual checksum. Corrupt or truncated file?","messagePattern":"Expected footer checksum did not match actual checksum\\. Corrupt or truncated file\\?","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/frame/file/FrameFileFooter.java","lineNumber":65,"sourceCode":"        footerMemory.getCapacity() - FrameFileWriter.TRAILER_LENGTH,\n        FrameFileWriter.TRAILER_LENGTH,\n        ByteOrder.LITTLE_ENDIAN\n    );\n    this.numFrames = trailer.getInt(0);\n    this.numPartitions = trailer.getInt(Integer.BYTES);\n    int length = trailer.getInt(Integer.BYTES * 2L);\n    int expectedFooterChecksum = trailer.getInt(Integer.BYTES * 3L);\n    // Verify footer begins with MARKER_NO_MORE_FRAMES.\n    if (footerMemory.getByte(0) != FrameFileWriter.MARKER_NO_MORE_FRAMES) {\n      throw new IOE(\"File [%s] end marker not in expected location\", \"file\");\n    }\n\n    // Verify footer checksum.\n    final int actualChecksum =\n        (int) footerMemory.xxHash64(0, footerMemory.getCapacity() - Integer.BYTES, FrameFileWriter.CHECKSUM_SEED);\n\n    if (expectedFooterChecksum != actualChecksum) {\n      throw new ISE(\"Expected footer checksum did not match actual checksum. Corrupt or truncated file?\");\n    }\n\n    // Verify footer length.\n    if (length != FrameFileWriter.footerLength(numFrames, numPartitions)) {\n      throw new ISE(\"Expected footer length did not match actual footer length. Corrupt or truncated file?\");\n    }\n  }\n\n  /**\n   * First frame of a given partition. Partitions beyond {@link #getNumPartitions()} are treated as empty: if provided,\n   * this method returns {@link #getNumFrames()}.\n   */\n  public int getPartitionStartFrame(final int partition)\n  {\n    if (partition < 0) {\n      throw new IAE(\"Partition [%,d] out of bounds\", partition);\n    } else if (partition >= numPartitions) {\n      // Frame might not have every partition, if some are empty.","sourceCodeStart":47,"sourceCodeEnd":83,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/frame/file/FrameFileFooter.java#L47-L83","documentation":"The footer carries an xxHash64 checksum of its own contents; FrameFileFooter recomputes it and compares. A mismatch means the footer bytes changed after writing — classic corruption/truncation — so the file cannot be trusted and reading is aborted.","triggerScenarios":"Opening a frame file whose footer bytes were altered (bit rot, partial write, edit after production) so the stored checksum no longer matches the recomputed hash.","commonSituations":"Disk/storage corruption; incomplete file transfer or download; file modified after task completion; concurrency where a reader sees a partially flushed file from a crashed writer.","solutions":["Treat the file as unrecoverable and re-run the job that produced the frame file.","Verify storage health if corruption recurs (disk errors, NFS/S3 consistency issues).","Confirm the file transfer used binary mode and completed fully.","Ensure only one process writes the file and readers wait for task completion."],"exampleFix":"// before\nFrameFile.open(suspectFile, maxMmapSize); // ISE on checksum mismatch\n// after\nif (checksumMatches(suspectFile, manifest.checksum())) {\n  FrameFile.open(suspectFile, maxMmapSize);\n} else {\n  regenerateFrameFile(manifest); // re-run producing task\n}","handlingStrategy":"try-catch","validationCode":"// verify the file end-to-end before use\ntry (java.io.RandomAccessFile raf = new java.io.RandomAccessFile(file, \"r\")) {\n  if (raf.length() != producerManifest.getSize()) throw new IllegalStateException(\"size mismatch\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  FrameFile.open(file, maxMmapSize);\n} catch (IllegalStateException e) {\n  if (e.getMessage().contains(\"checksum did not match\")) {\n    regenerateFrameFile(file); // corruption is not recoverable in place\n  } else {\n    throw e;\n  }\n}","preventionTips":["Keep checksums with produced frame files and verify after transfer/copy.","Watch for recurring disk/medium errors in the storage backend.","Ensure single-writer semantics — concurrent writers corrupt footers.","Avoid modifying files under the Druid segment/output directories."],"tags":["io","corruption","integrity","druid"],"backgroundTag":"checksum-mismatch","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}