{"record":{"id":"79f1e199938f87b0","repo":"apache/druid","slug":"invalid-frame-size-size-d-b","errorCode":null,"errorMessage":"Invalid frame size (size = %,d B)","messagePattern":"Invalid frame size \\(size = %,d B\\)","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"processing/src/main/java/org/apache/druid/frame/channel/ReadableByteChunksFrameChannel.java","lineNumber":427,"sourceCode":"    }\n\n    if (streamPart == StreamPart.FRAMES) {\n      if (bytesBuffered >= Byte.BYTES) {\n        final Memory memory = copyFromQueuedChunks(1);\n        final byte markerByte = memory.getByte(0);\n\n        if (markerByte == FrameFileWriter.MARKER_FRAME || markerByte == FrameFileWriter.MARKER_RAC) {\n          // Read nextFrameLength if needed; otherwise do nothing.\n          // Both MARKER_FRAME and MARKER_RAC use the same compression envelope format.\n          final int bytesRequiredToReadLength = FRAME_MARKER_BYTES + FrameCompression.COMPRESSED_DATA_HEADER_SIZE;\n\n          if (nextCompressedFrameLength == UNKNOWN_LENGTH && bytesBuffered >= bytesRequiredToReadLength) {\n            nextMarkerType = markerByte;\n            nextCompressedFrameLength = copyFromQueuedChunks(bytesRequiredToReadLength)\n                .getLong(FRAME_MARKER_BYTES + Byte.BYTES /* Compression strategy byte */);\n\n            if (nextCompressedFrameLength <= 0 || nextCompressedFrameLength >= MAX_FRAME_SIZE_BYTES) {\n              throw new ISE(\"Invalid frame size (size = %,d B)\", nextCompressedFrameLength);\n            }\n          }\n        } else if (markerByte == FrameFileWriter.MARKER_NO_MORE_FRAMES) {\n          streamPart = StreamPart.FOOTER;\n          nextCompressedFrameLength = UNKNOWN_LENGTH;\n          nextMarkerType = NO_MARKER;\n        } else {\n          throw new ISE(\"Invalid midstream marker (id = %s, position = %d)\", id, bytesAdded - bytesBuffered);\n        }\n      }\n    }\n\n    if (streamPart == StreamPart.FOOTER) {\n      if (bytesBuffered > 0) {\n        // Footer is discarded: it isn't useful when reading frame files as streams. (It contains pointers\n        // for random access of frames.)\n        deleteFromQueuedChunks(bytesBuffered);\n      }","sourceCodeStart":409,"sourceCodeEnd":445,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/processing/src/main/java/org/apache/druid/frame/channel/ReadableByteChunksFrameChannel.java#L409-L445","documentation":"When a frame marker is read, the 8-byte length following it must be positive and below MAX_FRAME_SIZE_BYTES. A zero, negative, or oversized length means the stream is corrupted or misaligned, so the reader throws instead of attempting an unsafe allocation. This guards against corrupt or desynchronized frame headers.","triggerScenarios":"Reading a frame length from the buffered header where nextCompressedFrameLength <= 0 or >= MAX_FRAME_SIZE_BYTES (approximately 2 GiB by default, the FrameFileWriter frame size limit).","commonSituations":"Frame files larger than the configured max frame size written by misconfigured producers; corrupted or truncated streams; reading at a wrong byte offset after a prior parse error; byte-order/format mismatch.","solutions":["Regenerate the frame data; check the writer's max frame size configuration.","Verify stream/file integrity (no truncation or overwrite).","Ensure reader and writer formats/versions match.","If length is consistently >= MAX_FRAME_SIZE_BYTES, confirm MAX_FRAME_SIZE_BYTES is not exceeded by data written by an older version with different limits."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { rac = channel.read(); } catch (ISE e) { if (e.getMessage().startsWith(\"Invalid frame size\")) { failStage(\"corrupt or oversized frame header\", e); } else { throw e; } }","preventionTips":["Keep frame sizes within MAX_FRAME_SIZE_BYTES on the producer side.","Regenerate spool files after any crash mid-write.","Avoid manual byte-level surgery on frame files."],"tags":["druid","frame-channel","corruption"],"backgroundTag":"value-out-of-range","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}