{"record":{"id":"cb4cec1f4a98c8c3","repo":"apache/druid","slug":"expected-footer-length-did-not-match-actual-footer","errorCode":null,"errorMessage":"Expected footer length did not match actual footer length. Corrupt or truncated file?","messagePattern":"Expected footer length did not match actual footer length\\. Corrupt or truncated file\\?","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/frame/file/FrameFileFooter.java","lineNumber":70,"sourceCode":"    this.numPartitions = trailer.getInt(Integer.BYTES);\n    int length = trailer.getInt(Integer.BYTES * 2L);\n    int expectedFooterChecksum = trailer.getInt(Integer.BYTES * 3L);\n    // Verify footer begins with MARKER_NO_MORE_FRAMES.\n    if (footerMemory.getByte(0) != FrameFileWriter.MARKER_NO_MORE_FRAMES) {\n      throw new IOE(\"File [%s] end marker not in expected location\", \"file\");\n    }\n\n    // Verify footer checksum.\n    final int actualChecksum =\n        (int) footerMemory.xxHash64(0, footerMemory.getCapacity() - Integer.BYTES, FrameFileWriter.CHECKSUM_SEED);\n\n    if (expectedFooterChecksum != actualChecksum) {\n      throw new ISE(\"Expected footer checksum did not match actual checksum. Corrupt or truncated file?\");\n    }\n\n    // Verify footer length.\n    if (length != FrameFileWriter.footerLength(numFrames, numPartitions)) {\n      throw new ISE(\"Expected footer length did not match actual footer length. Corrupt or truncated file?\");\n    }\n  }\n\n  /**\n   * First frame of a given partition. Partitions beyond {@link #getNumPartitions()} are treated as empty: if provided,\n   * this method returns {@link #getNumFrames()}.\n   */\n  public int getPartitionStartFrame(final int partition)\n  {\n    if (partition < 0) {\n      throw new IAE(\"Partition [%,d] out of bounds\", partition);\n    } else if (partition >= numPartitions) {\n      // Frame might not have every partition, if some are empty.\n      return numFrames;\n    } else {\n      final long partitionStartFrameLocation =\n          footerMemory.getCapacity()\n          - FrameFileWriter.TRAILER_LENGTH","sourceCodeStart":52,"sourceCodeEnd":88,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/frame/file/FrameFileFooter.java#L52-L88","documentation":"The footer records numFrames and numPartitions; the constructor recomputes the expected footer length with FrameFileWriter.footerLength and compares it to the footerLength stored in the trailer. A mismatch means the footer's declared structure does not match its declared size, indicating a corrupt or inconsistent file.","triggerScenarios":"Opening a frame file whose trailer footerLength disagrees with footerLength(numFrames, numPartitions) computed from the trailer — e.g. the trailer was overwritten, or the file mixes bytes from two different writer runs.","commonSituations":"Overlapping writes from a crashed and restarted task; corrupted disk sectors; files assembled incorrectly by external tooling; mixing Druid versions with different footer layouts.","solutions":["Regenerate the frame file; structural inconsistency in the footer is not repairable in place.","Check the producing task's logs for crash/restart during file writes and eliminate double-writing.","Ensure reader and writer Druid versions are compatible.","Verify storage/disk integrity if this happens repeatedly on the same volume."],"exampleFix":"// before\nFrameFile.open(mergedFile, maxMmapSize); // merged from partial files, inconsistent footer\n// after\nFile complete = jobOutput.getSingleCompleteFile(); // never stitch partial outputs\nFrameFile.open(complete, maxMmapSize);","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  FrameFile.open(file, maxMmapSize);\n} catch (IllegalStateException e) {\n  if (e.getMessage().contains(\"footer length did not match\")) {\n    throw new IllegalStateException(\"Frame file structurally inconsistent, rebuild: \" + file, e);\n  }\n  throw e;\n}","preventionTips":["Never merge or stitch partial frame-file outputs from failed tasks.","Ensure tasks do not restart and rewrite into the same output file path.","Use complete/atomic output publication (write-then-rename).","Keep writer/reader versions consistent."],"tags":["io","corruption","druid"],"backgroundTag":"checksum-mismatch","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}