apache/hadoop · error · InvalidJobConfException

Native-Task doesn't support sort class {}

Error message

Native-Task doesn't support sort class {}

What it means

The native map-output collector hard-codes an expectation that map.sort.class (Constants.MAP_SORT_CLASS) is QuickSort, the only IndexedSorter implementation its C++ buffer/sort protocol matches. init compares the configured class name to QuickSort's and throws InvalidJobConfException if any other sort implementation is set.

Source

Thrown at hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/java/org/apache/hadoop/mapred/nativetask/NativeMapOutputCollectorDelegator.java:101

      throw new InvalidJobConfException(message);
    }

    Class<?> comparatorClass = job.getClass(MRJobConfig.KEY_COMPARATOR, null,
        RawComparator.class);
    if (comparatorClass != null && !Platforms.define(comparatorClass)) {
      String message = "Native output collector doesn't support customized java comparator "
        + job.get(MRJobConfig.KEY_COMPARATOR);
      LOG.error(message);
      throw new InvalidJobConfException(message);
    }



    if (!QuickSort.class.getName().equals(job.get(Constants.MAP_SORT_CLASS))) {
      String message = "Native-Task doesn't support sort class " +
        job.get(Constants.MAP_SORT_CLASS);
      LOG.error(message);
      throw new InvalidJobConfException(message);
    }

    if (job.getBoolean(MRConfig.SHUFFLE_SSL_ENABLED_KEY, false) == true) {
      String message = "Native-Task doesn't support secure shuffle";
      LOG.error(message);
      throw new InvalidJobConfException(message);
    }

    final Class<?> keyCls = job.getMapOutputKeyClass();
    try {
      @SuppressWarnings("rawtypes")
      final INativeSerializer serializer = NativeSerialization.getInstance().getSerializer(keyCls);
      if (null == serializer) {
        String message = "Key type not supported. Cannot find serializer for " + keyCls.getName();
        LOG.error(message);
        throw new InvalidJobConfException(message);
      } else if (!Platforms.support(keyCls.getName(), serializer, job)) {
        String message = "Native output collector doesn't support this key, " +

View on GitHub (pinned to 2add963021)

Solutions

1. Unset map.sort.class so the default QuickSort applies, keeping the native collector
2. Or explicitly set map.sort.class to org.apache.hadoop.util.QuickSort
3. If the custom sort implementation is required, disable the native collector (unset mapreduce.job.map.output.collector.class) - the two are mutually exclusive

Example fix

# before
<property><name>map.sort.class</name><value>com.myco.Sorter</value></property>
<property><name>mapreduce.job.map.output.collector.class</name><value>org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator</value></property>

# after
<property><name>map.sort.class</name><value>org.apache.hadoop.util.QuickSort</value></property>

String sortClass = jobConf.get("map.sort.class");
if (sortClass != null && !QuickSort.class.getName().equals(sortClass)) {
  if ("org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator"
      .equals(jobConf.get("mapreduce.job.map.output.collector.class"))) {
    throw new IllegalArgumentException("nativetask requires QuickSort; unset map.sort.class");
  }
}

When it happens

Trigger: Setting map.sort.class (e.g. to a custom IndexedSorter or another pluggable-sort implementation from the PluggableShuffleAndPluggableSort framework) while the native output collector is enabled via mapreduce.job.map.output.collector.class.

Common situations: Clusters using pluggable shuffle/sort plugins (e.g. for alternative shuffle services) where someone additionally enables nativetask; experiments with custom sorters left in job configs carried over to a native-enabled environment.

Related errors

• Native output collector doesn't support customized java comp
• Native output collector doesn't support this key, this key i
• There is no reducer, no need to use native output collector
• Native-Task doesn't support secure shuffle
• Key type not supported. Cannot find serializer for {}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/72d917c73e999fa1. Report an issue: GitHub.