apache/hadoop · error · IOException

Image compression codec not supported: {codecClassName}

Error message

Image compression codec not supported: {codecClassName}

What it means

When a legacy fsimage records FSIMAGE_COMPRESSION=true, ImageLoaderCurrent reads the codec class name stored in the image and resolves it via CompressionCodecFactory.getCodecByClassName, which only sees codecs discoverable on the tool's classpath. A null return means no codec with that class name is registered, so the (compressed) remainder of the image cannot be decompressed and loading stops. This is a classpath/dependency problem, not image corruption.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/offlineImageViewer/ImageLoaderCurrent.java:208

        v.visit(ImageElement.SNAPSHOT_COUNTER, in.readInt());
        int numSnapshots = in.readInt();
        v.visit(ImageElement.NUM_SNAPSHOTS_TOTAL, numSnapshots);
        for (int i = 0; i < numSnapshots; i++) {
          processSnapshot(in, v);
        }
      }
      
      if (NameNodeLayoutVersion.supports(Feature.FSIMAGE_COMPRESSION, imageVersion)) {
        boolean isCompressed = in.readBoolean();
        v.visit(ImageElement.IS_COMPRESSED, String.valueOf(isCompressed));
        if (isCompressed) {
          String codecClassName = Text.readString(in);
          v.visit(ImageElement.COMPRESS_CODEC, codecClassName);
          CompressionCodecFactory codecFac = new CompressionCodecFactory(
              new Configuration());
          CompressionCodec codec = codecFac.getCodecByClassName(codecClassName);
          if (codec == null) {
            throw new IOException("Image compression codec not supported: "
                + codecClassName);
          }
          in = new DataInputStream(codec.createInputStream(in));
        }
      }
      processINodes(in, v, numInodes, skipBlocks, supportSnapshot);
      subtreeMap.clear();
      dirNodeMap.clear();

      processINodesUC(in, v, skipBlocks);

      if (NameNodeLayoutVersion.supports(Feature.DELEGATION_TOKEN, imageVersion)) {
        processDelegationTokens(in, v);
      }
      
      if (NameNodeLayoutVersion.supports(Feature.CACHING, imageVersion)) {
        processCacheManagerState(in, v);
      }

View on GitHub (pinned to 2add963021)

Solutions

1. Put the codec jar on the tool classpath: HADOOP_CLASSPATH=/path/to/hadoop-lzo.jar hdfs oiv_legacy -p Indented -i <fsimage> -o out.txt
2. Run oiv on a host with the full cluster classpath (same $HADOOP_HOME layout as the NameNode)
3. If the codec is unsupported site-wide, roll a fresh checkpoint with dfs.image.compress=false (or the default DefaultCodec/gzip) and process that image
4. Verify discovery beforehand: hadoop classpath | grep -i lzo

Example fix

# before
hdfs oiv_legacy -p Indented -i fsimage_lzo -o out.txt
# IOException: Image compression codec not supported: com.hadoop.compression.lzo.LzopCodec

# after
HADOOP_CLASSPATH=/opt/hadoop/lib/hadoop-lzo-0.4.20.jar \
  hdfs oiv_legacy -p Indented -i fsimage_lzo -o out.txt

// Confirm the image's codec is resolvable on THIS classpath before oiv runs
CompressionCodecFactory factory =
    new CompressionCodecFactory(new Configuration());
String codecClass = "com.hadoop.compression.lzo.LzopCodec"; // your dfs.image.compression.codec
if (factory.getCodecByClassName(codecClass) == null) {
  throw new IllegalStateException(
      "Codec " + codecClass + " not on classpath - add its jar via HADOOP_CLASSPATH");
}

try {
  ImageLoader loader = ImageLoader.loadImage(version);
  loader.loadImage(in, visitor, skipBlocks);
} catch (IOException e) {
  if (e.getMessage().startsWith("Image compression codec not supported")) {
    // recoverable by classpath fix, not by retrying the same command
    throw new IllegalStateException("Add the codec jar to HADOOP_CLASSPATH and rerun", e);
  }
  throw e;
}

When it happens

Trigger: `hdfs oiv_legacy` on an fsimage checkpointed with dfs.image.compression.codec set to a codec absent from the tool's classpath - classically LZO (com.hadoop.compression.lzo.LzopCodec), a site-custom codec, or Bzip2 with missing native libraries; also running oiv with a trimmed classpath (custom `hadoop jar ... -libjars`) that omits the codec jar.

Common situations: Clusters compressing checkpoints with LZO to save space; processing images on a gateway host that lacks hadoop-lzo; version skew where the codec jar moved or was renamed between Hadoop releases.

Related errors

• Unrecognized FSImage
• Unrecognized section {s.getName()}
• Unrecognized FSImage
• Too many distribution intervals {numIntervals}
• Cannot process fslayout version {imageVersion}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/46842663a5c6dd86. Report an issue: GitHub.