{"record":{"id":"8bc8fcb2e4a6cb6b","repo":"apache/hadoop","slug":"lz4-java-library-is-not-available-lz4decompressor","errorCode":null,"errorMessage":"lz4-java library is not available: Lz4Decompressor has not been loaded. You need to add lz4-java.jar to your CLASSPATH. {}","messagePattern":"lz4-java library is not available: Lz4Decompressor has not been loaded\\. You need to add lz4-java\\.jar to your CLASSPATH\\. (.+?)","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/lz4/Lz4Decompressor.java","lineNumber":63,"sourceCode":"  private byte[] userBuf = null;\n  private int userBufOff = 0, userBufLen = 0;\n  private boolean finished;\n\n  private LZ4SafeDecompressor lz4Decompressor;\n\n  /**\n   * Creates a new compressor.\n   *\n   * @param directBufferSize size of the direct buffer to be used.\n   */\n  public Lz4Decompressor(int directBufferSize) {\n    this.directBufferSize = directBufferSize;\n\n    try {\n      LZ4Factory lz4Factory = LZ4Factory.fastestInstance();\n      lz4Decompressor = lz4Factory.safeDecompressor();\n    } catch (AssertionError t) {\n      throw new RuntimeException(\"lz4-java library is not available: \" +\n              \"Lz4Decompressor has not been loaded. You need to add \" +\n              \"lz4-java.jar to your CLASSPATH. \" + t, t);\n    }\n\n    compressedDirectBuf = ByteBuffer.allocateDirect(directBufferSize);\n    uncompressedDirectBuf = ByteBuffer.allocateDirect(directBufferSize);\n    uncompressedDirectBuf.position(directBufferSize);\n\n  }\n\n  /**\n   * Creates a new decompressor with the default buffer size.\n   */\n  public Lz4Decompressor() {\n    this(DEFAULT_DIRECT_BUFFER_SIZE);\n  }\n\n  /**","sourceCodeStart":45,"sourceCodeEnd":81,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/compress/lz4/Lz4Decompressor.java#L45-L81","documentation":"Lz4Decompressor's constructor calls LZ4Factory.fastestInstance().safeDecompressor(); if the lz4-java runtime cannot produce any decompressor implementation (jar missing or unusable), the factory throws AssertionError, which is rethrown as RuntimeException with the 'add lz4-java.jar to your CLASSPATH' guidance. Same environmental failure as the compressor side, hitting the decompression path at object construction.","triggerScenarios":"new Lz4Decompressor(bufferSize) (directly or via Lz4Codec.createDecompressor()) when LZ4Factory.fastestInstance() throws AssertionError: no net.jpountz.lz4 classes resolvable, shaded-out natives, or a corrupted lz4-java jar on the classpath.","commonSituations":"Readers/MapReduce tasks on nodes with a different (slimmer) classpath than writers; user jars bundling an incompatible lz4-java shadowing Hadoop's; 'minimized' shaded clients (sqoop/flume custom sinks) dropping lz4 classes; upgrading hadoop-common without updating its compression dependencies.","solutions":["Ship/declare org.lz4:lz4-java (lz4-java.jar) on every node and client that constructs Lz4Decompressor, pinned to the version matching your Hadoop build.","Fix shading rules to include net.jpountz.lz4 classes and native libraries; verify with jar tf your-uber.jar | grep jpountz.","Remove duplicate/conflicting lz4 artifacts (mvn dependency:tree | grep lz4) so the factory finds one working implementation.","Validate codec availability at service startup by constructing the decompressor once in a health check."],"exampleFix":"// before\nDecompressor d = new Lz4Decompressor(64 * 1024); // RuntimeException, job dies mid-task\n\n// after\nstatic final boolean LZ4_PRESENT;\nstatic {\n  boolean ok;\n  try {\n    LZ4Factory.fastestInstance();\n    ok = true;\n  } catch (AssertionError | NoClassDefFoundError t) {\n    ok = false;\n  }\n  LZ4_PRESENT = ok;\n}\nDecompressor d = LZ4_PRESENT\n    ? new Lz4Decompressor(64 * 1024)\n    : codecFactory.getCodec(\"org.apache.hadoop.io.compress.DefaultCodec\").createDecompressor();","handlingStrategy":"validation","validationCode":"static boolean lz4DecompressorAvailable() {\n  try {\n    LZ4Factory.fastestInstance().safeDecompressor();\n    return true;\n  } catch (AssertionError | NoClassDefFoundError t) {\n    return false;\n  }\n}\n\nDecompressor d = lz4DecompressorAvailable()\n    ? new Lz4Decompressor(64 * 1024)\n    : fallbackCodec.createDecompressor();","typeGuard":null,"tryCatchPattern":"try {\n  return new Lz4Decompressor(directBufferSize);\n} catch (RuntimeException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"lz4-java library is not available\")) {\n    return fallbackCodec.createDecompressor(); // e.g. DefaultCodec path\n  }\n  throw e;\n}","preventionTips":["Ship lz4-java.jar on every node that reads, not just writes, lz4 data.","Pin lz4-java across writer and reader deployments to the same version.","Add a startup capability probe per node class (client/worker) so gaps surface at deploy time."],"tags":["lz4","compression","classpath","missing-dependency","hadoop","native-library"],"backgroundTag":"missing-classpath-dependency","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}