{"id":"ccaa75fa6530ac73","repo":"apache/kafka","slug":"dependent-block-stream-is-unsupported","errorCode":null,"errorMessage":"Dependent block stream is unsupported","messagePattern":"Dependent block stream is unsupported","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/common/compress/Lz4BlockOutputStream.java","lineNumber":318,"sourceCode":"            return new FLG(reserved,\n                           contentChecksum,\n                           contentSize,\n                           blockChecksum,\n                           blockIndependence,\n                           version);\n        }\n\n        public byte toByte() {\n            return (byte) (((reserved & 3) << 0) | ((contentChecksum & 1) << 2)\n                    | ((contentSize & 1) << 3) | ((blockChecksum & 1) << 4) | ((blockIndependence & 1) << 5) | ((version & 3) << 6));\n        }\n\n        private void validate() {\n            if (reserved != 0) {\n                throw new RuntimeException(\"Reserved bits must be 0\");\n            }\n            if (blockIndependence != 1) {\n                throw new RuntimeException(\"Dependent block stream is unsupported\");\n            }\n            if (version != VERSION) {\n                throw new RuntimeException(String.format(\"Version %d is unsupported\", version));\n            }\n        }\n\n        public boolean isContentChecksumSet() {\n            return contentChecksum == 1;\n        }\n\n        public boolean isContentSizeSet() {\n            return contentSize == 1;\n        }\n\n        public boolean isBlockChecksumSet() {\n            return blockChecksum == 1;\n        }\n","sourceCodeStart":300,"sourceCodeEnd":336,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/common/compress/Lz4BlockOutputStream.java#L300-L336","documentation":"Thrown by FLG.validate() when the Block-Independence bit (bit 5 of FLG) is 0, meaning the producer requested a 'dependent block stream' where each block reuses the previous block's dictionary. Kafka's Lz4BlockInputStream/OutputStream only implement independent-block mode (the LZ4 frame default), so a dependent-block frame is rejected.","triggerScenarios":"FLG.fromByte(b) where bit 5 == 0, typically while reading a frame produced by an LZ4 tool configured for linked/block-dependent mode. Reached from readHeader() at line 127. The Kafka producer always writes blockIndependence=1 (see FLG public constructor line 274).","commonSituations":"Reading Kafka records compressed by a non-Kafka LZ4 producer (e.g. lz4 CLI with --block-dependency or a streaming library that defaults to linked blocks); ingesting external LZ4 data through a custom Kafka serializer; consuming data that was re-compressed by an intermediary using block-dependent mode.","solutions":["Re-compress the data using Kafka's CompressionType.LZ4 (or any producer emitting blockIndependence=1).","If you must consume external LZ4 data, decompress it with a general-purpose LZ4 library that supports linked-block mode before it reaches Kafka's Lz4BlockInputStream.","Inspect the FLG byte: bit 5 must be 1 for Kafka compatibility."],"exampleFix":"// before: external lz4 CLI wrote linked-block frame\n//   lz4 --block-dependency file.dat file.dat.lz4\n//   -> consumed via Kafka custom serializer -> throws\n\n// after: produce the frame in independent-block mode\n//   lz4 file.dat file.dat.lz4    (default is independent)\n// or in code, use Kafka's own compressor so blockIndependence is always 1:","handlingStrategy":"try-catch","validationCode":"// FLG.blockIndependence is bit 5; Kafka only emits block-independence=1.\nbyte flgByte = frameBuf.get(flgOffset);\nif ((flgByte & 0x20) == 0) {\n    throw new IOException(\"Refusing LZ4 frame: dependent-block mode is not supported by Kafka\");\n}","typeGuard":"null","tryCatchPattern":"try {\n    try (Lz4BlockInputStream in = new Lz4BlockInputStream(payload, BufferSupplier.NO_CACHING, false)) {\n        // consume\n    }\n} catch (RuntimeException e) {\n    if (e.getMessage() != null && e.getMessage().contains(\"Dependent block stream is unsupported\")) {\n        // Producer used a non-Kafka LZ4 encoder in dependent-block mode; reject the record.\n        log.warn(\"Rejecting LZ4 frame produced in dependent-block mode on {}\", tp);\n        return;\n    }\n    throw e;\n}","preventionTips":["Only produce records with Kafka's compression (CompressionType.LZ4 via producer config); third-party LZ4 encoders may emit dependent-block frames Kafka cannot read.","If bridging external LZ4 data into Kafka, re-compress it on the producer side rather than passing raw frames through.","Document the LZ4 frame constraints (independent blocks, version 1) at any custom integration boundary."],"tags":["compression","lz4","frame-format","validation","kafka-clients"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}