apache/hadoop · error · IllegalArgumentException

Codec not configured for custom codec {}

Error message

Codec not configured for custom codec {}

What it means

Thrown by CodecUtil.getCodecClassName (reached via CodecUtil.createEncoder / createDecoder) when the schema's codec name is not one of the built-in cases in the switch ("rs", "rs-legacy", "xor", "hhxor"). For any other name Hadoop falls into the custom-codec path, which requires the Configuration key "io.erasurecode.codec.<codec>.coder" to be set to a class name; the class is later loaded and must be a subclass of org.apache.hadoop.io.erasurecode.codec.ErasureCodec with a (Configuration, ErasureCodecOptions) constructor. A missing key means the codec name is unrecognized and unconfigured, so an IllegalArgumentException is thrown before any coding happens.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/erasurecode/CodecUtil.java:280

    case ErasureCodeConstants.RS_LEGACY_CODEC_NAME:
      //TODO:rs-legacy should be handled differently.
      return conf.get(
          CodecUtil.IO_ERASURECODE_CODEC_RS_KEY,
          CodecUtil.IO_ERASURECODE_CODEC_RS);
    case ErasureCodeConstants.XOR_CODEC_NAME:
      return conf.get(
          CodecUtil.IO_ERASURECODE_CODEC_XOR_KEY,
          CodecUtil.IO_ERASURECODE_CODEC_XOR);
    case ErasureCodeConstants.HHXOR_CODEC_NAME:
      return conf.get(
          CodecUtil.IO_ERASURECODE_CODEC_HHXOR_KEY,
          CodecUtil.IO_ERASURECODE_CODEC_HHXOR);
    default:
      // For custom codec, we throw exception if the factory is not configured
      String codecKey = "io.erasurecode.codec." + codec + ".coder";
      String codecClass = conf.get(codecKey);
      if (codecClass == null) {
        throw new IllegalArgumentException("Codec not configured " +
                "for custom codec " + codec);
      }
      return codecClass;
    }
  }
}

View on GitHub (pinned to 2add963021)

Solutions

1. If a standard codec was intended, change the schema's codec name to exactly "rs", "rs-legacy", "xor", or "hhxor" (case-sensitive, no surrounding whitespace)
2. For a genuinely custom codec, set conf property "io.erasurecode.codec.<codec>.coder" to the canonical class name of your ErasureCodec implementation on every process that builds coders (NameNode for policy handling, DataNodes for actual EC work)
3. Verify the effective configuration in the failing process with conf.get("io.erasurecode.codec.<name>.coder") and confirm the class is on the classpath
4. Ensure the configured class extends ErasureCodec and exposes a public (Configuration, ErasureCodecOptions) constructor

Example fix

// before
Configuration conf = new Configuration();
// schema uses codec name "mycodec" but nothing is configured
ErasureEncoder enc = CodecUtil.createEncoder(conf, options); // IllegalArgumentException

// after
conf.set("io.erasurecode.codec.mycodec.coder",
    "com.example.codec.MyErasureCodec"); // class extends ErasureCodec
ErasureEncoder enc = CodecUtil.createEncoder(conf, options);

String codec = schema.getCodecName();
boolean builtin = codec.equals("rs") || codec.equals("rs-legacy")
    || codec.equals("xor") || codec.equals("hhxor");
if (!builtin && conf.get("io.erasurecode.codec." + codec + ".coder") == null) {
  throw new IllegalStateException("Custom codec '" + codec
      + "' has no io.erasurecode.codec." + codec + ".coder property");
}
ErasureEncoder enc = CodecUtil.createEncoder(conf, options);

try {
  encoder = CodecUtil.createEncoder(conf, options);
} catch (IllegalArgumentException e) {
  // message names the codec; surface a config hint to operators
  throw new ConfigurationException("EC codec '" + codecName
      + "' not configured: set io.erasurecode.codec." + codecName + ".coder", e);
}

When it happens

Trigger: Calling CodecUtil.createEncoder(conf, options) or CodecUtil.createDecoder(conf, options) where options.getSchema().getCodecName() is a custom or misspelled name (e.g. "RS", "rs ", "mycodec") while conf has no "io.erasurecode.codec.<name>.coder" entry. In HDFS this happens when a user-defined EC policy schema is applied but the codec class property was never added to the NameNode/DataNode configuration.

Common situations: Deploying a site-specific ErasureCodec on only some nodes; typo or wrong case in the codec name of an EC policy schema; policy files copied between clusters whose configuration does not define the custom coder key; Hadoop upgrades where codec naming conventions changed.

Related errors

• Can not create a Path from a null string
• Can not create a Path from an empty string
• mkdirs path arg is null
• Errors on getting mount table loader class. The fs.viewfs.mo
• %s=null: %s: %s

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/27d68c2e8d87a8eb. Report an issue: GitHub.