{"record":{"id":"ce5706395f21b2a2","repo":"apache/hadoop","slug":"native-output-collector-doesn-t-support-this-key","errorCode":null,"errorMessage":"Native output collector doesn't support this key, this key is not comparable in native: {}","messagePattern":"Native output collector doesn't support this key, this key is not comparable in native: (.+?)","errorType":"exception","errorClass":"InvalidJobConfException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/java/org/apache/hadoop/mapred/nativetask/NativeMapOutputCollectorDelegator.java","lineNumber":122,"sourceCode":"    if (job.getBoolean(MRConfig.SHUFFLE_SSL_ENABLED_KEY, false) == true) {\n      String message = \"Native-Task doesn't support secure shuffle\";\n      LOG.error(message);\n      throw new InvalidJobConfException(message);\n    }\n\n    final Class<?> keyCls = job.getMapOutputKeyClass();\n    try {\n      @SuppressWarnings(\"rawtypes\")\n      final INativeSerializer serializer = NativeSerialization.getInstance().getSerializer(keyCls);\n      if (null == serializer) {\n        String message = \"Key type not supported. Cannot find serializer for \" + keyCls.getName();\n        LOG.error(message);\n        throw new InvalidJobConfException(message);\n      } else if (!Platforms.support(keyCls.getName(), serializer, job)) {\n        String message = \"Native output collector doesn't support this key, \" +\n          \"this key is not comparable in native: \" + keyCls.getName();\n        LOG.error(message);\n        throw new InvalidJobConfException(message);\n      }\n    } catch (final IOException e) {\n      String message = \"Cannot find serializer for \" + keyCls.getName();\n      LOG.error(message);\n      throw new IOException(message);\n    }\n\n    final boolean ret = NativeRuntime.isNativeLibraryLoaded();\n    if (ret) {\n      if (job.getBoolean(MRJobConfig.MAP_OUTPUT_COMPRESS, false)) {\n        String codec = job.get(MRJobConfig.MAP_OUTPUT_COMPRESS_CODEC);\n        if (!NativeRuntime.supportsCompressionCodec(codec.getBytes(StandardCharsets.UTF_8))) {\n          String message = \"Native output collector doesn't support compression codec \" + codec;\n          LOG.error(message);\n          throw new InvalidJobConfException(message);\n        }\n      }\n      NativeRuntime.configure(job);","sourceCodeStart":104,"sourceCodeEnd":140,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/java/org/apache/hadoop/mapred/nativetask/NativeMapOutputCollectorDelegator.java#L104-L140","documentation":"Even when a serializer exists for the key class, the native collector must also compare keys in C++ to sort and spill. init calls Platforms.support(keyClassName, serializer, job); a false return means the key is serializable but not comparable by the native platform, and init throws InvalidJobConfException('...not comparable in native: <class>'). This separates 'cannot serialize' from 'cannot compare' failures.","triggerScenarios":"A map output key class has a registered native serializer but the active native platform has no comparator for it - e.g. a custom WritableComparable handled by a serialization-only platform - with the native collector enabled.","commonSituations":"Partially ported custom platforms that add a serializer but forget the native comparator; composite/custom WritableComparable keys on native-enabled clusters where only standard types compare natively.","solutions":["Switch the map output key to a fully supported type (Text, IntWritable, LongWritable, FloatWritable, DoubleWritable, BytesWritable) whose comparison the platform implements natively","Complete the custom platform: implement the native comparator for the key class so Platforms.support returns true","Disable the native collector for this job (unset mapreduce.job.map.output.collector.class)"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"INativeSerializer serializer = NativeSerialization.getInstance().getSerializer(keyCls);\nif (serializer != null && !Platforms.support(keyCls.getName(), serializer, job)) {\n  jobConf.unset(\"mapreduce.job.map.output.collector.class\"); // key not comparable natively\n}","typeGuard":null,"tryCatchPattern":"try {\n  collector.init(context);\n} catch (InvalidJobConfException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"not comparable in native\")) {\n    // pick a fully supported key type or fall back to the Java collector\n  }\n  throw e;\n}","preventionTips":["Test custom keys against both getSerializer (serializable) and Platforms.support (comparable) before rollout","Complete native platform implementations: a serializer without a comparator is not enough"],"tags":["nativetask","serialization","comparable","sorting","job-configuration"],"backgroundTag":"unsupported-key-serialization","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}