{"record":{"id":"83bc7977c3267a1f","repo":"apache/hadoop","slug":"native-output-collector-cannot-be-loaded","errorCode":null,"errorMessage":"Native output collector cannot be loaded;","messagePattern":"Native output collector cannot be loaded;","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/java/org/apache/hadoop/mapred/nativetask/NativeMapOutputCollectorDelegator.java","lineNumber":165,"sourceCode":"    } else {\n      String message = \"NativeRuntime cannot be loaded, please check that \" +\n        \"libnativetask.so is in hadoop library dir\";\n      LOG.error(message);\n      throw new InvalidJobConfException(message);\n    }\n\n    this.handler = null;\n    try {\n      final Class<K> oKClass = (Class<K>) job.getMapOutputKeyClass();\n      final Class<K> oVClass = (Class<K>) job.getMapOutputValueClass();\n      final TaskAttemptID id = context.getMapTask().getTaskID();\n      final TaskContext taskContext = new TaskContext(job, null, null, oKClass, oVClass,\n          context.getReporter(), id);\n      handler = NativeCollectorOnlyHandler.create(taskContext);\n    } catch (final IOException e) {\n      String message = \"Native output collector cannot be loaded;\";\n      LOG.error(message);\n      throw new IOException(message, e);\n    }\n\n    LOG.info(\"Native output collector can be successfully enabled!\");\n  }\n\n}\n","sourceCodeStart":147,"sourceCodeEnd":172,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-mapreduce-project/hadoop-mapreduce-client/hadoop-mapreduce-client-nativetask/src/main/java/org/apache/hadoop/mapred/nativetask/NativeMapOutputCollectorDelegator.java#L147-L172","documentation":"After all checks pass, init builds a TaskContext and calls NativeCollectorOnlyHandler.create(taskContext), which sets up the native collector (buffers, handler objects, JNI calls). If that throws an IOException, init re-throws it as IOException('Native output collector cannot be loaded;') with the original exception chained as the cause. On success you see 'Native output collector can be successfully enabled!'.","triggerScenarios":"Any IOException inside NativeCollectorOnlyHandler.create during map-task startup: native object creation failing at handler setup, buffer allocation problems, or downstream configuration errors - with the native library loaded and all job-conf gates already passed.","commonSituations":"Memory-constrained nodes failing native buffer allocation; partially working native installs where earlier checks pass but specific handler creation fails; version drift between Java protocol constants and the C++ library.","solutions":["Read the chained cause (getCause()) in the task log - it carries the actual failure; this outer message is only a wrapper","Verify jar/so version alignment and rerun 'hadoop checknative'","Disable the native collector for the job (unset mapreduce.job.map.output.collector.class) to restore throughput while investigating"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  collector.init(context);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().startsWith(\"Native output collector cannot be loaded\")) {\n    Throwable root = e.getCause(); // actual failure is chained here\n    LOG.error(\"native collector setup failed\", root);\n    // decide: fix env per root cause, or rerun with Java collector\n  }\n  throw e;\n}","preventionTips":["Always inspect getCause() for this wrapper - the outer message has no detail","Keep jar/so versions aligned and checknative-clean before enabling nativetask","Watch for the success line 'Native output collector can be successfully enabled!' in healthy runs"],"tags":["nativetask","jni","error-chaining","initialization"],"backgroundTag":"native-handler-initialization-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}