juicedata/juicefs · error · RuntimeException

Compression codec not found:

Error message

Compression codec not found: 

What it means

TestDFSIO benchmark wraps any failure to instantiate the configured compression codec in a RuntimeException with the (unfortunately empty, due to string concatenation bug) message "Compression codec not found: ". The code resolves the compression config value to a class via Class.forName and checks it is a CompressionCodec subclass; any exception (class not on classpath, wrong name, not a codec) triggers this.

Source

Thrown at sdk/java/src/main/java/io/juicefs/bench/TestDFSIO.java:492

      buffer = ThreadLocal.withInitial(() -> new byte[bufferSize]);
      try {
        baseDir = conf.get("test.basedir");
        fs = new Path(baseDir).getFileSystem(conf);
      } catch (IOException e) {
        throw new RuntimeException("Cannot create file system.", e);
      }
      randomBytes = conf.getBoolean("test.randomBytes", false);

      // grab compression
      String compression = getConf().get("test.io.compression.class", null);
      Class<? extends CompressionCodec> codec;

      // try to initialize codec
      try {
        codec = (compression == null) ? null :
                Class.forName(compression).asSubclass(CompressionCodec.class);
      } catch (Exception e) {
        throw new RuntimeException("Compression codec not found: ", e);
      }

      if (codec != null) {
        compressionCodec = (CompressionCodec)
                ReflectionUtils.newInstance(codec, getConf());
      }

    }

    Path getDataDir() {
      return new Path(baseDir, "io_data");
    }

    @Override
      // IOMapperBase
    void collectStats(OutputCollector<Text, Text> output,
                      String name,
                      long execTime,

View on GitHub (pinned to c9a67b23e8)

Solutions

1. Use the fully-qualified class name, e.g. org.apache.hadoop.io.compress.GzipCodec, in the -compression argument
2. Verify the codec jar is on the classpath (HADOOP_CLASSPATH or lib dir)
3. Omit the -compression option to disable compression (codec = null)
4. Add the codec to io.compression.codecs and verify with `hadoop jar ...` before benchmarking

Example fix

// before
compression = "gzip"
// after
compression = "org.apache.hadoop.io.compress.GzipCodec"

try {
  Class<?> cls = Class.forName(compression);
  if (!CompressionCodec.class.isAssignableFrom(cls))
    throw new IllegalArgumentException(compression + " is not a CompressionCodec");
} catch (ClassNotFoundException e) {
  throw new IllegalArgumentException("Codec not on classpath: " + compression, e);
}

static boolean isValidCodec(String name, Configuration conf) {
  try {
    Class<?> c = Class.forName(name);
    return CompressionCodec.class.isAssignableFrom(c);
  } catch (Exception e) { return false; }
}

try {
  runBenchmark(conf);
} catch (RuntimeException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("Compression codec not found")) {
    LOG.error("Fix -compression: must be FQCN of a CompressionCodec on the classpath");
  } else throw e;
}

When it happens

Trigger: Running TestDFSIO with -compression set to a class name that does not exist on the classpath, is misspelled, or is not a CompressionCodec subclass; configure() is called during localRun setup.

Common situations: Typo in codec class name (e.g. org.apache.hadoop.io.compress.GzipCompressor vs GzipCodec); missing jar on Hadoop classpath; using a codec name string like 'gzip' instead of a fully-qualified class name (this code requires FQCN, unlike core-site io.compression.codecs).

Related errors

• Hadoop version was incompatible, current hadoop version is:\
• construct fileStatus failed
• incompatible hadoop version
• Invalid start or len parameter
• stream was closed

AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06). Data as JSON: /api/errors/65497bf76d3c83cb. Report an issue: GitHub.