{"record":{"id":"72d917c73e999fa1","repo":"apache/hadoop","slug":"native-task-doesn-t-support-sort-class","errorCode":null,"errorMessage":"Native-Task doesn't support sort class {}","messagePattern":"Native-Task doesn't support sort class (.+?)","errorType":"exception","errorClass":"InvalidJobConfException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/java/org/apache/hadoop/mapred/nativetask/NativeMapOutputCollectorDelegator.java","lineNumber":101,"sourceCode":"      throw new InvalidJobConfException(message);\n    }\n\n    Class<?> comparatorClass = job.getClass(MRJobConfig.KEY_COMPARATOR, null,\n        RawComparator.class);\n    if (comparatorClass != null && !Platforms.define(comparatorClass)) {\n      String message = \"Native output collector doesn't support customized java comparator \"\n        + job.get(MRJobConfig.KEY_COMPARATOR);\n      LOG.error(message);\n      throw new InvalidJobConfException(message);\n    }\n\n\n\n    if (!QuickSort.class.getName().equals(job.get(Constants.MAP_SORT_CLASS))) {\n      String message = \"Native-Task doesn't support sort class \" +\n        job.get(Constants.MAP_SORT_CLASS);\n      LOG.error(message);\n      throw new InvalidJobConfException(message);\n    }\n\n    if (job.getBoolean(MRConfig.SHUFFLE_SSL_ENABLED_KEY, false) == true) {\n      String message = \"Native-Task doesn't support secure shuffle\";\n      LOG.error(message);\n      throw new InvalidJobConfException(message);\n    }\n\n    final Class<?> keyCls = job.getMapOutputKeyClass();\n    try {\n      @SuppressWarnings(\"rawtypes\")\n      final INativeSerializer serializer = NativeSerialization.getInstance().getSerializer(keyCls);\n      if (null == serializer) {\n        String message = \"Key type not supported. Cannot find serializer for \" + keyCls.getName();\n        LOG.error(message);\n        throw new InvalidJobConfException(message);\n      } else if (!Platforms.support(keyCls.getName(), serializer, job)) {\n        String message = \"Native output collector doesn't support this key, \" +","sourceCodeStart":83,"sourceCodeEnd":119,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/java/org/apache/hadoop/mapred/nativetask/NativeMapOutputCollectorDelegator.java#L83-L119","documentation":"The native map-output collector hard-codes an expectation that map.sort.class (Constants.MAP_SORT_CLASS) is QuickSort, the only IndexedSorter implementation its C++ buffer/sort protocol matches. init compares the configured class name to QuickSort's and throws InvalidJobConfException if any other sort implementation is set.","triggerScenarios":"Setting map.sort.class (e.g. to a custom IndexedSorter or another pluggable-sort implementation from the PluggableShuffleAndPluggableSort framework) while the native output collector is enabled via mapreduce.job.map.output.collector.class.","commonSituations":"Clusters using pluggable shuffle/sort plugins (e.g. for alternative shuffle services) where someone additionally enables nativetask; experiments with custom sorters left in job configs carried over to a native-enabled environment.","solutions":["Unset map.sort.class so the default QuickSort applies, keeping the native collector","Or explicitly set map.sort.class to org.apache.hadoop.util.QuickSort","If the custom sort implementation is required, disable the native collector (unset mapreduce.job.map.output.collector.class) - the two are mutually exclusive"],"exampleFix":"# before\n<property><name>map.sort.class</name><value>com.myco.Sorter</value></property>\n<property><name>mapreduce.job.map.output.collector.class</name><value>org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator</value></property>\n\n# after\n<property><name>map.sort.class</name><value>org.apache.hadoop.util.QuickSort</value></property>","handlingStrategy":"validation","validationCode":"String sortClass = jobConf.get(\"map.sort.class\");\nif (sortClass != null && !QuickSort.class.getName().equals(sortClass)) {\n  if (\"org.apache.hadoop.mapred.nativetask.NativeMapOutputCollectorDelegator\"\n      .equals(jobConf.get(\"mapreduce.job.map.output.collector.class\"))) {\n    throw new IllegalArgumentException(\"nativetask requires QuickSort; unset map.sort.class\");\n  }\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Never combine pluggable sort implementations with the native collector","Leave map.sort.class unset unless a custom sorter is genuinely required"],"tags":["nativetask","sorting","job-configuration","pluggable-framework"],"backgroundTag":"native-task-unsupported-job-config","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}