{"record":{"id":"67a3e273ddd4e08a","repo":"apache/druid","slug":"cannot-read-frame-of-size-d-bytes","errorCode":null,"errorMessage":"Cannot read frame of size [%,d] bytes","messagePattern":"Cannot read frame of size \\[%,d\\] bytes","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/frame/channel/ReadableByteChunksFrameChannel.java","lineNumber":360,"sourceCode":"  long getBytesBuffered()\n  {\n    synchronized (lock) {\n      return bytesBuffered;\n    }\n  }\n\n  private RowsAndColumns nextRAC()\n  {\n    final Memory compressedMemory;\n    final byte markerType;\n\n    synchronized (lock) {\n      if (!canReadFrame()) {\n        throw new ISE(\"Frame of size [%,d] not yet ready to read\", nextCompressedFrameLength);\n      }\n\n      if (nextCompressedFrameLength > Integer.MAX_VALUE - FRAME_MARKER_BYTES - FrameCompression.COMPRESSED_DATA_ENVELOPE_SIZE) {\n        throw new ISE(\"Cannot read frame of size [%,d] bytes\", nextCompressedFrameLength);\n      }\n\n      markerType = nextMarkerType;\n      final int numBytes = Ints.checkedCast(FRAME_MARKER_AND_COMPRESSED_ENVELOPE_BYTES + nextCompressedFrameLength);\n      compressedMemory = copyFromQueuedChunks(numBytes).region(\n          FRAME_MARKER_BYTES,\n          FRAME_MARKER_AND_COMPRESSED_ENVELOPE_BYTES + nextCompressedFrameLength - FRAME_MARKER_BYTES\n      );\n      deleteFromQueuedChunks(numBytes);\n      updateStreamState();\n    }\n\n    final byte[] decompressedBytes =\n        FrameCompression.decompress(compressedMemory, 0, compressedMemory.getCapacity());\n\n    switch (markerType) {\n      case FrameFileWriter.MARKER_FRAME:\n        final Frame frame = Frame.wrap(decompressedBytes);","sourceCodeStart":342,"sourceCodeEnd":378,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/frame/channel/ReadableByteChunksFrameChannel.java#L342-L378","documentation":"During frame decoding, the announced compressed frame length exceeds the maximum representable payload that can fit in an int-sized buffer after subtracting marker and compression-envelope overhead. This guard prevents an absurd or corrupted length field from being used to allocate/copy a huge buffer. It almost always indicates corrupted stream data or a bad length read from the stream.","triggerScenarios":"nextCompressedFrameLength (read from 8 bytes of the buffered frame header) is greater than Integer.MAX_VALUE - FRAME_MARKER_BYTES - FrameCompression.COMPRESSED_DATA_ENVELOPE_SIZE, i.e. a corrupt or misaligned frame header.","commonSituations":"Corrupted spooled frame files or inter-process streams (disk corruption, truncated/overwritten files); reading a stream written by an incompatible Druid version with a different frame format; stream desynchronization after earlier mis-parse.","solutions":["Verify the frame file/stream was produced by the same Druid version and format (FrameFileWriter MAGIC check passes).","Re-run the stage/query to regenerate the frame data; inspect underlying storage for corruption.","Check that the file was not truncated or modified after writing (compare sizes/checksums).","If reproducible, capture the id and frame header bytes and report with the full query log."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// regenerate data; no pre-call check available to users beyond trusting source integrity\nif (!frameFile.exists() || frameFile.length() == 0) { throw new IllegalStateException(\"missing/incomplete frame file\"); }","typeGuard":null,"tryCatchPattern":"try { rac = channel.read(); } catch (ISE e) { if (e.getMessage().startsWith(\"Cannot read frame of size\")) { failStage(\"corrupted frame stream\", e); } else { throw e; } }","preventionTips":["Run on reliable storage; verify spool directories are not shared or cleaned concurrently.","Pin all tasks in a stage to the same Druid version.","Fail fast on truncation: check file sizes before reading."],"tags":["druid","frame-channel","corruption"],"backgroundTag":"schema-validation-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}