{"record":{"id":"17917afca8f8507d","repo":"apache/hadoop","slug":"bailing-out-since-native-library-couldn-t-be-loade","errorCode":null,"errorMessage":"Bailing out since native library couldn't be loaded","messagePattern":"Bailing out since native library couldn't be loaded","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/JniBasedUnixGroupsMapping.java","lineNumber":49,"sourceCode":"import org.apache.hadoop.util.NativeCodeLoader;\nimport org.slf4j.Logger;\nimport org.slf4j.LoggerFactory;\n\n/**\n * A JNI-based implementation of {@link GroupMappingServiceProvider} \n * that invokes libC calls to get the group\n * memberships of a given user.\n */\n@InterfaceAudience.LimitedPrivate({\"HDFS\", \"MapReduce\"})\n@InterfaceStability.Evolving\npublic class JniBasedUnixGroupsMapping implements GroupMappingServiceProvider {\n  \n  private static final Logger LOG =\n      LoggerFactory.getLogger(JniBasedUnixGroupsMapping.class);\n\n  static {\n    if (!NativeCodeLoader.isNativeCodeLoaded()) {\n      throw new RuntimeException(\"Bailing out since native library couldn't \" +\n        \"be loaded\");\n    }\n    anchorNative();\n    LOG.debug(\"Using JniBasedUnixGroupsMapping for Group resolution\");\n  }\n\n  /**\n   * Set up our JNI resources.\n   *\n   * @throws                 RuntimeException if setup fails.\n   */\n  native static void anchorNative();\n\n  /**\n   * Get the set of groups associated with a user.\n   *\n   * @param username           The user name\n   *","sourceCodeStart":31,"sourceCodeEnd":67,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/JniBasedUnixGroupsMapping.java#L31-L67","documentation":"JniBasedUnixGroupsMapping shells out to C via JNI to resolve Unix groups. Its static initializer requires NativeCodeLoader.isNativeCodeLoaded(); if the Hadoop native library (libhadoop.so) is not loaded it throws RuntimeException and class initialization fails. Because the class is instantiated reflectively from hadoop.security.group.mapping, the visible symptom in logs is usually an ExceptionInInitializerError / 'Failed to load new group mapping class' followed by fallback or startup failure.","triggerScenarios":"hadoop.security.group.mapping=org.apache.hadoop.security.JniBasedUnixGroupsMapping (often via the default JniBasedUnixGroupsMappingWithFallback going wrong) while libhadoop.so is absent for the platform, LD_LIBRARY_PATH/HADOOP_HOME/lib/native lacks it, glibc is incompatible, or the JVM is 64-bit with only 32-bit natives.","commonSituations":"Running on exotic/archaic OSes without prebuilt natives; LD_LIBRARY_PATH not propagated to daemon environments; HADOOP_OPTS missing -Djava.library.path; macOS/arm or Alpine musl where bundled .so won't load; downgrading/upgrading Hadoop leaving stale native dirs.","solutions":["Check the startup log for the actual native-load failure ('Unable to load native-hadoop library') and fix the root cause: correct java.library.path/LD_LIBRARY_PATH to $HADOOP_HOME/lib/native","Compile or download the native library matching your platform (mvn package -Pnative -DskipTests in hadoop-common, or distro-specific package)","Switch hadoop.security.group.mapping to org.apache.hadoop.security.ShellBasedUnixGroupsMapping (pure Java, runs id -Gn) when natives are unavailable","Verify glibc/architecture compatibility (file libhadoop.so, ldd output) and that HADOOP_OPTS carries the library path into daemon JVMs"],"exampleFix":"# before (core-site.xml)\n<property><name>hadoop.security.group.mapping</name>\n  <value>org.apache.hadoop.security.JniBasedUnixGroupsMapping</value></property>\n\n# after: fallback-safe mapping without natives\n<property><name>hadoop.security.group.mapping</name>\n  <value>org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback</value></property>\n# or pure-shell\n<property><name>hadoop.security.group.mapping</name>\n  <value>org.apache.hadoop.security.ShellBasedUnixGroupsMapping</value></property>\n\n# and ensure natives are found\nexport HADOOP_OPTS=\"$HADOOP_OPTS -Djava.library.path=$HADOOP_HOME/lib/native\"","handlingStrategy":"fallback","validationCode":"if (conf.get(\"hadoop.security.group.mapping\", \"\")\n        .contains(\"JniBasedUnixGroupsMapping\")\n    && !NativeCodeLoader.isNativeCodeLoaded()) {\n  LOG.warn(\"Native library not loaded; JNI group mapping will fail. \"\n      + \"Falling back to ShellBasedUnixGroupsMapping.\");\n  conf.set(\"hadoop.security.group.mapping\",\n      \"org.apache.hadoop.security.ShellBasedUnixGroupsMapping\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  Class<?> c = Class.forName(\"org.apache.hadoop.security.JniBasedUnixGroupsMapping\");\n  GroupMappingServiceProvider m = (GroupMappingServiceProvider) c.newInstance();\n} catch (ExceptionInInitializerError | NoClassDefFoundError e) {\n  // static init failed: natives missing — switch mapping implementation\n  LOG.warn(\"JNI group mapping unavailable: {}\", e.toString());\n}","preventionTips":["Verify natives at rollout: run `hadoop checknative` on every node","Use ...WithFallback mapping classes so resolution degrades to shell instead of failing","Propagate java.library.path/LD_LIBRARY_PATH into daemon environments explicitly"],"tags":["native-library","jni","group-mapping","classpath","hadoop"],"backgroundTag":"native-library-load-failed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}