{"id":"5ee846396695a03c","repo":"apache/kafka","slug":"stream-ended-prematurely","errorCode":null,"errorMessage":"Stream ended prematurely","messagePattern":"Stream ended prematurely","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/common/compress/Lz4BlockInputStream.java","lineNumber":118,"sourceCode":"\n    /**\n     * Check whether KafkaLZ4BlockInputStream is configured to ignore the\n     * Frame Descriptor checksum, which is useful for compatibility with\n     * old client implementations that use incorrect checksum calculations.\n     */\n    public boolean ignoreFlagDescriptorChecksum() {\n        return this.ignoreFlagDescriptorChecksum;\n    }\n\n    /**\n     * Reads the magic number and frame descriptor from input buffer.\n     *\n     * @throws IOException\n     */\n    private void readHeader() throws IOException {\n        // read first 6 bytes into buffer to check magic and FLG/BD descriptor flags\n        if (in.remaining() < 6) {\n            throw new IOException(PREMATURE_EOS);\n        }\n\n        if (MAGIC != in.getInt()) {\n            throw new IOException(NOT_SUPPORTED);\n        }\n        // mark start of data to checksum\n        in.mark();\n\n        flg = FLG.fromByte(in.get());\n        maxBlockSize = BD.fromByte(in.get()).getBlockMaximumSize();\n\n        if (flg.isContentSizeSet()) {\n            if (in.remaining() < 8) {\n                throw new IOException(PREMATURE_EOS);\n            }\n            in.position(in.position() + 8);\n        }\n","sourceCodeStart":100,"sourceCodeEnd":136,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/common/compress/Lz4BlockInputStream.java#L100-L136","documentation":"Thrown in Lz4BlockInputStream.readHeader() when the input ByteBuffer has fewer than 6 bytes remaining at the start of a frame, meaning it cannot even contain the 4-byte magic plus FLG/BD descriptor bytes of an LZ4 frame. It signals a truncated/corrupt LZ4 frame header and is raised as IOException(PREMATURE_EOS). This is the earliest possible failure when decompressing a record batch whose LZ4 payload was cut off.","triggerScenarios":"Constructing/reading an Lz4BlockInputStream over a ByteBuffer with <6 bytes; happens during Kafka record-batch decompression when a compressed record batch is truncated by a network error, partial file, log corruption, or when the buffer passed in only contains a fragment of the LZ4 frame.","commonSituations":"Truncated log segment files after a broker crash or disk fault; partial messages from a producer/consumer over an unstable connection; mis-split buffers before passing them to the decompressor; custom serdes that slice a record batch incorrectly.","solutions":["Verify the source ByteBuffer contains a complete LZ4 frame before decompression (at minimum the header plus one block and end mark).","Check upstream transport/storage for truncation: network/replication errors, partial writes, or filesystem corruption.","If you sliced the buffer manually, ensure position/limit correctly bound the full compressed payload and weren't advanced past the start.","Reproduce with a known-good batch to rule out producer-side corruption."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"// Header read needs at least 6 bytes; truncated stream surfaces as IOException\ntry {\n    try (Lz4BlockInputStream in = new Lz4BlockInputStream(buffer, ignoreFlagDescriptorChecksum)) {\n        // ... read fully ...\n    }\n} catch (IOException e) {\n    // \"Stream ended prematurely\" -- source buffer is shorter than the LZ4 frame header;\n    // record offset, skip the record, do not retry the same payload\n}","preventionTips":["Treat any decompression IOException as evidence of message corruption or truncation, not a transient fault.","Rely on Kafka's per-message CRC/checksum (and producer compression config) to catch truncation before decompression.","When reading from byte[], confirm buffer length >= 6 before wrapping; for streaming sources, ensure framing delivers complete records.","Log the topic-partition-offset alongside the failure so corrupted batches are locatable and skippable."],"tags":["compression","lz4","io","corruption","java"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}