{"id":"09e95f58fd0b0fc3","repo":"apache/kafka","slug":"kafka-has-detected-a-buggy-lz4-java-library-1-4","errorCode":null,"errorMessage":"Kafka has detected a buggy lz4-java library (< 1.4.x) on the classpath. If you are using Kafka client libraries, make sure your application does not accidentally override the version provided by Kafka or include multiple versions of the library on the classpath. The lz4-java version on the classpath should match the version the Kafka client libraries depend on. Adding -verbose:class to your JVM arguments may help understand which lz4-java version is getting loaded.","messagePattern":"Kafka has detected a buggy lz4-java library \\(< 1\\.4\\.x\\) on the classpath\\. If you are using Kafka client libraries, make sure your application does not accidentally override the version provided by Kafka or include multiple versions of the library on the classpath\\. The lz4-java version on the classpath should match the version the Kafka client libraries depend on\\. Adding -verbose:class to your JVM arguments may help understand which lz4-java version is getting loaded\\.","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"critical","filePath":"clients/src/main/java/org/apache/kafka/common/compress/Lz4BlockInputStream.java","lineNumber":305,"sourceCode":"        final byte[] compressed = new byte[compressor.maxCompressedLength(source.length)];\n        final int compressedLength = compressor.compress(source, 0, source.length, compressed, 0,\n                                                         compressed.length);\n\n        // allocate an array-backed ByteBuffer with non-zero array-offset containing the compressed data\n        // a buggy decompressor will read the data from the beginning of the underlying array instead of\n        // the beginning of the ByteBuffer, failing to decompress the invalid data.\n        final byte[] zeroes = {0, 0, 0, 0, 0};\n        ByteBuffer nonZeroOffsetBuffer = ByteBuffer\n            .allocate(zeroes.length + compressed.length) // allocates the backing array with extra space to offset the data\n            .put(zeroes) // prepend invalid bytes (zeros) before the compressed data in the array\n            .slice() // create a new ByteBuffer sharing the underlying array, offset to start on the compressed data\n            .put(compressed); // write the compressed data at the beginning of this new buffer\n\n        ByteBuffer dest = ByteBuffer.allocate(source.length);\n        try {\n            DECOMPRESSOR.decompress(nonZeroOffsetBuffer, 0, compressedLength, dest, 0, source.length);\n        } catch (Exception e) {\n            throw new RuntimeException(\"Kafka has detected a buggy lz4-java library (< 1.4.x) on the classpath.\"\n                                       + \" If you are using Kafka client libraries, make sure your application does not\"\n                                       + \" accidentally override the version provided by Kafka or include multiple versions\"\n                                       + \" of the library on the classpath. The lz4-java version on the classpath should\"\n                                       + \" match the version the Kafka client libraries depend on. Adding -verbose:class\"\n                                       + \" to your JVM arguments may help understand which lz4-java version is getting loaded.\", e);\n        }\n    }\n}\n","sourceCodeStart":287,"sourceCodeEnd":314,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/common/compress/Lz4BlockInputStream.java#L287-L314","documentation":"Thrown from the static initializer of Lz4BlockInputStream when detectBrokenLz4Version() catches any exception while decompressing a known-good array-backed ByteBuffer with a non-zero array offset. lz4-java versions earlier than 1.4.x have a bug (lz4/lz4-java#65) where the decompressor reads from the start of the backing array instead of the buffer's array-offset, producing garbage; Kafka refuses to run with such a version because it would corrupt decompression. The exception wraps the underlying LZ4Exception as its cause.","triggerScenarios":"Class-loading Lz4BlockInputStream for the first time in the JVM triggers the static block (lines 58-66); if the buggy lz4-java is on the classpath, detectBrokenLz4Version() at line 303 throws and the wrapped RuntimeException is re-thrown from the constructor (line 91) every time an instance is created.","commonSituations":"Application bundles an old lz4-java (e.g. 1.3.x) via a transitive dependency (Hadoop, Spark, Cassandra, Elasticsearch, Flink, an old HBase client); a fat-jar/shaded-jar overrides the Kafka-provided lz4-java; multiple lz4-java JARs on the classpath with the older winning classloading; explicit dependency pinning in pom.xml/gradle that forgets to bump lz4-java. Common after upgrading the Kafka client but not cleaning caches of older deps.","solutions":["Run mvn dependency:tree / gradle dependencies and force lz4-java to the version declared by the Kafka client you use (check clients/build.gradle for the exact lz4-java version).","Remove duplicate/older lz4-java JARs from the classpath (shade plugin exclude, <dependencyManagement> enforcement, or drop the bundled copy).","Add -verbose:class to JVM args to identify exactly which lz4-java JAR and version is being loaded.","If shading, ensure the shade plugin does not split lz4-java's native bindings from its Java classes (keep net.jpountz.* and the JNI loader together)."],"exampleFix":"// before (pom.xml) — unmanaged, stale transitive lz4-java pulled in by another lib\n<dependency>\n  <groupId>org.apache.kafka</groupId>\n  <artifactId>kafka-clients</artifactId>\n  <version>3.7.0</version>\n</dependency>\n\n// after — pin lz4-java to the Kafka-blessed version for the whole build\n<dependencyManagement>\n  <dependencies>\n    <dependency>\n      <groupId>org.lz4</groupId>\n      <artifactId>lz4-java</artifactId>\n      <version>1.8.0</version>\n    </dependency>\n  </dependencies>\n</dependencyManagement>","handlingStrategy":"validation","validationCode":"// Detect the broken lz4-java (< 1.4.x) before constructing any LZ4 stream.\n// Mirror Kafka's own probe at app startup.\nimport net.jpountz.lz4.LZ4Factory;\nimport java.util.regex.Matcher;\nimport java.util.regex.Pattern;\nString ver = LZ4Factory.fastestInstance().fastCompressor().toString(); // or read Package\nPackage lz4Pkg = LZ4Factory.class.getPackage();\nString spec = lz4Pkg != null ? lz4Pkg.getImplementationVersion() : null;\n// fail fast with a clear message instead of triggering Kafka's nested exception:\nif (spec == null || compareAtLeast(spec, \"1.4.0\") < 0) {\n    throw new IllegalStateException(\"lz4-java >= 1.4.0 required, found: \" + spec);\n}","typeGuard":"null","tryCatchPattern":"// Construction of Lz4BlockInputStream triggers the static BROKEN_LZ4 check.\ntry {\n    new Lz4BlockInputStream(buf, BufferSupplier.NO_CACHING, true);\n} catch (RuntimeException e) {\n    if (e.getMessage() != null && e.getMessage().contains(\"buggy lz4-java library\")) {\n        // halt the JVM with an actionable dependency error rather than retrying\n        System.err.println(e.getMessage());\n        System.exit(1);\n    }\n    throw e;\n}","preventionTips":["Let Kafka clients own the lz4-java version — do not declare your own org.lz4:lz4-java dependency unless you pin it to exactly the version the Kafka client release uses.","Enable the Maven Enforcer Plugin (enforce / dependencyConvergence) or Gradle's strictly version constraints so transitive lz4-java versions cannot drift.","Run the Kafka client integration smoke test (produce + consume one LZ4-compressed record) in CI to catch classpath regressions before deploy.","Use -verbose:class (or jcmd ClassName.all) in staging to confirm which lz4-java JAR is actually loaded at runtime.","Shaded/fat JARs are the usual culprit — re-resolve and re-pin lz4-java after any shading step."],"tags":["compression","lz4","classpath","dependency","version-mismatch","kafka-clients"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}