apache/hadoop · error · InvalidJobConfException

Key type not supported. Cannot find serializer for {}

Error message

Key type not supported. Cannot find serializer for {}

What it means

The native collector serializes map-output keys with native code, so the map output key class must have a registered INativeSerializer. init calls NativeSerialization.getInstance().getSerializer(keyCls); a null return (no serializer factory registered for that class) produces InvalidJobConfException('Key type not supported. Cannot find serializer for <class>'). Out of the box, native serializers exist for the standard Writable key types (Text, IntWritable, LongWritable, FloatWritable, DoubleWritable, BytesWritable) plus whatever platforms add.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/java/org/apache/hadoop/mapred/nativetask/NativeMapOutputCollectorDelegator.java:117

        job.get(Constants.MAP_SORT_CLASS);
      LOG.error(message);
      throw new InvalidJobConfException(message);
    }

    if (job.getBoolean(MRConfig.SHUFFLE_SSL_ENABLED_KEY, false) == true) {
      String message = "Native-Task doesn't support secure shuffle";
      LOG.error(message);
      throw new InvalidJobConfException(message);
    }

    final Class<?> keyCls = job.getMapOutputKeyClass();
    try {
      @SuppressWarnings("rawtypes")
      final INativeSerializer serializer = NativeSerialization.getInstance().getSerializer(keyCls);
      if (null == serializer) {
        String message = "Key type not supported. Cannot find serializer for " + keyCls.getName();
        LOG.error(message);
        throw new InvalidJobConfException(message);
      } else if (!Platforms.support(keyCls.getName(), serializer, job)) {
        String message = "Native output collector doesn't support this key, " +
          "this key is not comparable in native: " + keyCls.getName();
        LOG.error(message);
        throw new InvalidJobConfException(message);
      }
    } catch (final IOException e) {
      String message = "Cannot find serializer for " + keyCls.getName();
      LOG.error(message);
      throw new IOException(message);
    }

    final boolean ret = NativeRuntime.isNativeLibraryLoaded();
    if (ret) {
      if (job.getBoolean(MRJobConfig.MAP_OUTPUT_COMPRESS, false)) {
        String codec = job.get(MRJobConfig.MAP_OUTPUT_COMPRESS_CODEC);
        if (!NativeRuntime.supportsCompressionCodec(codec.getBytes(StandardCharsets.UTF_8))) {
          String message = "Native output collector doesn't support compression codec " + codec;

View on GitHub (pinned to 2add963021)

Solutions

1. Use a natively supported key type (Text, IntWritable, LongWritable, FloatWritable, DoubleWritable, BytesWritable) as the map output key
2. Implement and register a custom INativeSerializer via a nativetask platform for your key class (NativeSerialization.register)
3. Disable the native collector for this job by unsetting mapreduce.job.map.output.collector.class

Example fix

// before
job.setMapOutputKeyClass(MyCompositeKey.class); // custom Writable
conf.set("mapreduce.job.map.output.collector.class",
    "org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator");

// after
job.setMapOutputKeyClass(Text.class); // serialize composite key as delimited Text

Class<?> keyCls = job.getMapOutputKeyClass();
boolean nativeCollector = "org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator"
    .equals(jobConf.get("mapreduce.job.map.output.collector.class"));
if (nativeCollector
    && NativeSerialization.getInstance().getSerializer(keyCls) == null) {
  jobConf.unset("mapreduce.job.map.output.collector.class"); // no native serializer for this key
}

try {
  collector.init(context);
} catch (InvalidJobConfException e) {
  if (e.getMessage() != null && e.getMessage().startsWith("Key type not supported")) {
    // switch map output key to a natively supported Writable, or disable native collector
  }
  throw e;
}

When it happens

Trigger: Setting mapreduce.map.output.key.class (or job.setMapOutputKeyClass) to a custom or non-standard Writable while the native collector is enabled; getSerializer finds no factory for the class and returns null.

Common situations: Jobs with domain-specific composite key Writables moved onto a native-enabled cluster; using Avro/Protobuf-based key wrappers, or third-party Writable types, as the map output key.

Related errors

• Native output collector doesn't support this key, this key i
• Cannot find serializer for {}
• Exception while get content summary
• f.toString()
• Class {} already registered but maps to {} and not {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/2c87d4cd31c639a9. Report an issue: GitHub.