apache/hadoop · error · IllegalArgumentException

Unsupported compression algorithm name:

Error message

Unsupported compression algorithm name: 

What it means

Thrown by Compression.getCompressionAlgorithmByName(String) when the argument matches no Algorithm enum constant (lzo, gz, snappy, none). TFile uses this lookup to translate a compression name into an algorithm, so any unrecognized string is rejected with IllegalArgumentException before a writer is created. Note that a known-but-unsupported name (e.g. "lzo" without the codec) does NOT fail here; only a misspelled or unknown name does.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/file/tfile/Compression.java:368

        CodecPool.returnDecompressor(decompressor);
      }
    }

    public String getName() {
      return compressName;
    }
  }

  public static Algorithm getCompressionAlgorithmByName(String compressName) {
    Algorithm[] algos = Algorithm.class.getEnumConstants();

    for (Algorithm a : algos) {
      if (a.getName().equals(compressName)) {
        return a;
      }
    }

    throw new IllegalArgumentException(
        "Unsupported compression algorithm name: " + compressName);
  }

  static String[] getSupportedAlgorithms() {
    Algorithm[] algos = Algorithm.class.getEnumConstants();

    ArrayList<String> ret = new ArrayList<String>();
    for (Algorithm a : algos) {
      if (a.isSupported()) {
        ret.add(a.getName());
      }
    }
    return ret.toArray(new String[ret.size()]);
  }
}

View on GitHub (pinned to 2add963021)

Solutions

1. Validate the name against TFile.getSupportedCompressionAlgorithms() (or the Algorithm enum) before using it
2. Use the TFile constants (TFile.COMPRESSION_GZ, TFile.COMPRESSION_LZO, TFile.COMPRESSION_SNAPPY, TFile.COMPRESSION_NONE) instead of string literals
3. Normalize external input: trim and lowercase it, and map known aliases such as "gzip"/"zlib" to "gz"

Example fix

// before
String name = jobConfig.get("my.compression", "gzip");
TFile.Writer w = new TFile.Writer(out, size, name, null);

// after
String name = jobConfig.get("my.compression", TFile.COMPRESSION_GZ);
if (!Arrays.asList(TFile.getSupportedCompressionAlgorithms()).contains(name)) {
  throw new IllegalArgumentException("Unknown compression: " + name);
}
TFile.Writer w = new TFile.Writer(out, size, name, null);

// Whitelist before use
Set<String> ok = new HashSet<>(Arrays.asList(TFile.getSupportedCompressionAlgorithms()));
if (!ok.contains(name)) {
  throw new IllegalArgumentException(
      "compression '" + name + "' not recognized; valid: " + ok);
}
Compression.Algorithm algo = Compression.getCompressionAlgorithmByName(name);

catch (IllegalArgumentException e) {
  // report the valid names from TFile.getSupportedCompressionAlgorithms() to the caller
}

When it happens

Trigger: Passing a user- or config-supplied compression string to new TFile.Writer(...) or directly to Compression.getCompressionAlgorithmByName() when it is not exactly one of the enum names: "gzip", "GZ", "zlib", "snappy4", "" are all rejected.

Common situations: Configuration files reusing gzip-style names ("gzip" instead of "gz"); CLI arguments or job settings with typos or mixed case; code ported from another library whose codec names differ; version drift where an algorithm name was renamed or removed.

Related errors

• LZO codec %s=%s could not be loaded
• LZO codec class not specified. Did you forget to set propert
• Non-alphanumeric data found in input, aborting.
• Compression option provided does not match the file
• All merged files must be compressed or not.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/736d8f988d454b60. Report an issue: GitHub.