{"record":{"id":"5a0db286ae8dff3d","repo":"apache/hadoop","slug":"lzo-codec-class-not-specified-did-you-forget-to-s","errorCode":null,"errorMessage":"LZO codec class not specified. Did you forget to set property io.compression.codec.lzo.class?","messagePattern":"LZO codec class not specified\\. Did you forget to set property io\\.compression\\.codec\\.lzo\\.class\\?","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/Compression.java","lineNumber":129,"sourceCode":"      }\n\n      @Override\n      CompressionCodec getCodec() throws IOException {\n        if (!isSupported()) {\n          throw new IOException(String.format(\n              \"LZO codec %s=%s could not be loaded\", CONF_LZO_CLASS, clazz),\n                  cnf);\n        }\n\n        return codec;\n      }\n\n      @Override\n      public synchronized InputStream createDecompressionStream(\n          InputStream downStream, Decompressor decompressor,\n          int downStreamBufferSize) throws IOException {\n        if (!isSupported()) {\n          throw new IOException(\n              \"LZO codec class not specified. Did you forget to set property \"\n                  + CONF_LZO_CLASS + \"?\");\n        }\n        InputStream bis1 = null;\n        if (downStreamBufferSize > 0) {\n          bis1 = new BufferedInputStream(downStream, downStreamBufferSize);\n        } else {\n          bis1 = downStream;\n        }\n        conf.setInt(IO_COMPRESSION_CODEC_LZO_BUFFERSIZE_KEY,\n            IO_COMPRESSION_CODEC_LZO_BUFFERSIZE_DEFAULT);\n        CompressionInputStream cis =\n            codec.createInputStream(bis1, decompressor);\n        BufferedInputStream bis2 = new BufferedInputStream(cis, DATA_IBUF_SIZE);\n        return bis2;\n      }\n\n      @Override","sourceCodeStart":111,"sourceCodeEnd":147,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/Compression.java#L111-L147","documentation":"Thrown by Compression.Algorithm.LZO.createDecompressionStream() when a TFile data block compressed with LZO must be decompressed but Algorithm.LZO.isSupported() returns false, i.e. no usable LZO codec was loaded. Despite the wording, the property io.compression.codec.lzo.class always has a default value, so in practice this means the configured codec class failed to load (usually ClassNotFoundException) rather than that it was never specified. Reading an LZO-compressed TFile is impossible without the codec.","triggerScenarios":"Opening a TFile.Reader over a file written with the \"lzo\" algorithm (the reader builds a decompression stream per data block) while io.compression.codec.lzo.class resolves to a class that is not loadable on the current JVM.","commonSituations":"Reading LZO-compressed TFiles or old HFile/sequence data on a cluster that lacks the hadoop-lzo jar; a task or client node whose classpath excludes the codec jar; a codec class name set for a different hadoop-lago/hadoop-lzo version; moving files from an LZO-enabled cluster to one without it.","solutions":["Install the matching hadoop-lzo jar (and native library) on the node doing the read and retry","Check the effective value of io.compression.codec.lzo.class (Configuration and system property) and correct it to the installed codec class","Copy the file to a host that has the codec and rewrite it with \"gz\" or \"none\" via TFile before reading cluster-wide"],"exampleFix":"// before\nTFile.Reader reader = new TFile.Reader(hfileIn, TFile.RFS_LENGTH, conf); // file written with \"lzo\"\n\n// after\nif (!Compression.Algorithm.LZO.isSupported()) {\n  throw new IOException(\"Cannot read LZO TFile: hadoop-lzo codec not installed\");\n}\nTFile.Reader reader = new TFile.Reader(hfileIn, TFile.RFS_LENGTH, conf);","handlingStrategy":"validation","validationCode":"// Before opening a reader over a possibly-LZO file\nString algo = TFile.getCompressionAlgorithmByName(reader.getCompressionName());\nif (algo.equals(TFile.COMPRESSION_LZO) && !Compression.Algorithm.LZO.isSupported()) {\n  throw new IOException(\"File is LZO-compressed but no LZO codec is installed on this host\");\n}","typeGuard":null,"tryCatchPattern":"catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"LZO codec\")) {\n    // decompression impossible without deployment fix: surface to operator\n  }\n}","preventionTips":["Standardize on gz or none for new files unless LZO is a hard requirement","Record the compression algorithm in file names or sidecar metadata so readers can pre-check codec availability","Verify codec availability on the exact node class that performs the read (task nodes, not just the client)"],"tags":["hadoop","tfile","lzo","compression","codec-unavailable"],"backgroundTag":"optional-dependency-missing","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}