{"record":{"id":"9e6543e4cfe01734","repo":"apache/druid","slug":"file-s-is-too-short-size-d","errorCode":null,"errorMessage":"File[%s] is too short (size[%,d])","messagePattern":"File\\[(.+?)\\] is too short \\(size\\[%,d\\]\\)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/frame/file/FrameFile.java","lineNumber":198,"sourceCode":"  ) throws IOException\n  {\n    final EnumSet<Flag> flagSet = flags.length == 0 ? EnumSet.noneOf(Flag.class) : EnumSet.copyOf(Arrays.asList(flags));\n\n    if (!file.exists()) {\n      throw new FileNotFoundException(StringUtils.format(\"File[%s] not found\", file));\n    }\n\n    // Closer for mmap that is shared across all references: either footer only (if file size is larger\n    // Integer.MAX_VALUE) or entire file (if file size is smaller than, or equal to, Integer.MAX_VALUE).\n    Closeable sharedMapCloser = null;\n\n    try (final RandomAccessFile randomAccessFile = new RandomAccessFile(file, \"r\")) {\n      final long fileLength = randomAccessFile.length();\n\n      // Verify minimum file length.\n      if (fileLength <\n          FrameFileWriter.MAGIC.length + FrameFileWriter.TRAILER_LENGTH + Byte.BYTES /* MARKER_NO_MORE_FRAMES */) {\n        throw new IOE(\"File[%s] is too short (size[%,d])\", file, fileLength);\n      }\n\n      // Verify magic.\n      final byte[] buf = new byte[FrameFileWriter.TRAILER_LENGTH /* Larger than FrameFileWriter.MAGIC */];\n      final Memory bufMemory = Memory.wrap(buf, ByteOrder.LITTLE_ENDIAN);\n      randomAccessFile.readFully(buf, 0, FrameFileWriter.MAGIC.length);\n\n      if (!bufMemory.equalTo(0, Memory.wrap(FrameFileWriter.MAGIC), 0, FrameFileWriter.MAGIC.length)) {\n        throw new IOE(\"File[%s] is not a frame file\", file);\n      }\n\n      // Read number of frames and partitions.\n      randomAccessFile.seek(fileLength - FrameFileWriter.TRAILER_LENGTH);\n      randomAccessFile.readFully(buf, 0, FrameFileWriter.TRAILER_LENGTH);\n\n      final int footerLength = bufMemory.getInt(Integer.BYTES * 2L);\n      if (footerLength < 0) {\n        throw new ISE(\"Negative-size footer. Corrupt or truncated file[%s]?\", file);","sourceCodeStart":180,"sourceCodeEnd":216,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/frame/file/FrameFile.java#L180-L216","documentation":"FrameFile.open() validates the minimum length of a frame file: the magic bytes plus the trailer plus the no-more-frames marker byte. A file shorter than this cannot possibly contain a valid header and footer, so open fails with an IOE reporting the actual size. This guards against reading empty or truncated frame files.","triggerScenarios":"Opening a frame file whose size is smaller than FrameFileWriter.MAGIC.length + TRAILER_LENGTH + 1 bytes — zero-byte or partially written frame files, e.g. a writer crashed mid-flush or the file was truncated by storage.","commonSituations":"Durable storage truncated frame files (disk full, crash during write); downstream stage reading frames before the writer finished flushing; copying frames with size limits (e.g. tools that cap file size).","solutions":["Regenerate the frame by re-running the stage/query that wrote it.","Check free disk space and writer logs for incomplete flushes or crashes.","Skip/ignore the corrupt partial file if the writer is idempotent and will rewrite it.","Verify any transfer/copy step preserves full file size."],"exampleFix":"// before\nFrameFile.open(new File(partitionPath)); // File[...] is too short (size[0])\n\n// after\nif (new File(partitionPath).length() < 64) { // below MIN_FRAME_FILE_LENGTH\n  throw new IllegalStateException(\"Incomplete frame partition, re-run stage: \" + partitionPath);\n}\nFrameFile.open(new File(partitionPath));","handlingStrategy":"validation","validationCode":"// Java: sanity-check size before opening\nfinal long MIN_FRAME_FILE_SIZE = 64; // MAGIC + TRAILER + marker\nif (frameFile.length() < MIN_FRAME_FILE_SIZE) {\n  throw new IOException(\"Frame file incomplete, regenerating: \" + frameFile);\n}","typeGuard":null,"tryCatchPattern":"try {\n  FrameFile.open(file, flags);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"is too short\")) {\n    throw new RetryableQueryException(\"Truncated frame file, re-run stage: \" + file, e);\n  }\n  throw e;\n}","preventionTips":["Ensure writers complete and fsync frame files before downstream stages read them.","Monitor disk space to prevent truncated writes on durable storage.","Verify copy/transfer steps preserve full file sizes."],"tags":["frame-file","file-too-short","corruption","io","druid"],"backgroundTag":"file-too-short","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}