{"record":{"id":"dbc73f6f9fe9b1de","repo":"apache/hadoop","slug":"lzo-codec-s-s-could-not-be-loaded","errorCode":null,"errorMessage":"LZO codec %s=%s could not be loaded","messagePattern":"LZO codec (.+?)=(.+?) could not be loaded","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/Compression.java","lineNumber":116,"sourceCode":"          checked = true;\n          reinitCodecInTests = conf.getBoolean(\"test.reload.lzo.codec\", false);\n          clazz = getLzoCodecClass();\n          try {\n            LOG.info(\"Trying to load Lzo codec class: \" + clazz);\n            codec =\n                (CompressionCodec) ReflectionUtils.newInstance(Class\n                    .forName(clazz), conf);\n          } catch (ClassNotFoundException e) {\n            cnf = e;\n          }\n        }\n        return codec != null;\n      }\n\n      @Override\n      CompressionCodec getCodec() throws IOException {\n        if (!isSupported()) {\n          throw new IOException(String.format(\n              \"LZO codec %s=%s could not be loaded\", CONF_LZO_CLASS, clazz),\n                  cnf);\n        }\n\n        return codec;\n      }\n\n      @Override\n      public synchronized InputStream createDecompressionStream(\n          InputStream downStream, Decompressor decompressor,\n          int downStreamBufferSize) throws IOException {\n        if (!isSupported()) {\n          throw new IOException(\n              \"LZO codec class not specified. Did you forget to set property \"\n                  + CONF_LZO_CLASS + \"?\");\n        }\n        InputStream bis1 = null;\n        if (downStreamBufferSize > 0) {","sourceCodeStart":98,"sourceCodeEnd":134,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/Compression.java#L98-L134","documentation":"Thrown by Compression.Algorithm.LZO.getCodec() when the LZO codec class cannot be loaded. The class name comes from the configuration key io.compression.codec.lzo.class (or a JVM system property of the same name), defaulting to org.apache.hadoop.io.compress.LzoCodec. Apache Hadoop cannot ship an LZO implementation because LZO is GPL licensed, so the class is only present when a third-party hadoop-lzo jar is installed. The original ClassNotFoundException is chained as the cause and names the exact missing class.","triggerScenarios":"Creating a TFile.Writer or opening a TFile.Reader with compression name \"lzo\"; calling Compression.Algorithm.LZO.getCodec(), createCompressionStream(), or createDecompressionStream() when Class.forName() on the configured codec class throws ClassNotFoundException because io.compression.codec.lzo.class (or its default) is not on the classpath.","commonSituations":"Using a vanilla Apache Hadoop distribution that has no hadoop-lzo jar; io.compression.codec.lzo.class pointing at a class from an incompatible hadoop-lzo version; the codec jar present on the cluster but missing from the client or task classpath; reading data files produced on another cluster that had LZO installed.","solutions":["Install a third-party LZO implementation (e.g. the kevinweil/hadoop-lzo jar plus its native library) on every node and client, and add it to the classpath","If you use a different LZO codec class, set io.compression.codec.lzo.class in the Configuration (or as a JVM system property) to its fully qualified name, then verify with Compression.Algorithm.LZO.isSupported()","If LZO is not required, write and read the TFile with \"gz\", \"snappy\", or \"none\" instead","If the file must remain LZO, read it on a host that has the codec and re-compress it to a supported algorithm"],"exampleFix":"// before\nTFile.Writer writer = new TFile.Writer(fout, TFile.MIN_BLOCK_SIZE, TFile.COMPRESSION_LZO, null);\n\n// after\nString algo = Compression.Algorithm.LZO.isSupported()\n    ? TFile.COMPRESSION_LZO : TFile.COMPRESSION_GZ;\nTFile.Writer writer = new TFile.Writer(fout, TFile.MIN_BLOCK_SIZE, algo, null);","handlingStrategy":"validation","validationCode":"// Probe once at startup before any LZO TFile work\nif (!Compression.Algorithm.LZO.isSupported()) {\n  throw new IllegalStateException(\n      \"LZO codec class (io.compression.codec.lzo.class=\"\n      + \". Defaults to org.apache.hadoop.io.compress.LzoCodec) is not on the classpath. \"\n      + \"Install hadoop-lzo or choose another compression algorithm.\");\n}","typeGuard":null,"tryCatchPattern":"catch (IOException e) {\n  if (e.getCause() instanceof ClassNotFoundException) {\n    // codec class missing: fail fast with a deployment message, do not retry\n  } else { throw e; }\n}","preventionTips":["Run Algorithm.LZO.isSupported() as a startup self-check on every node role that reads or writes LZO TFiles","Ship the hadoop-lzo jar and native library via the same deployment mechanism as Hadoop itself so versions stay in sync","Centralize the compression-algorithm choice in one config key validated at job submission"],"tags":["hadoop","tfile","lzo","compression","classpath"],"backgroundTag":"class-not-found","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}