apache/hadoop · error · RuntimeException

Bailing out since native library couldn't be loaded

Error message

Bailing out since native library couldn't be loaded

What it means

JniBasedUnixGroupsMapping shells out to C via JNI to resolve Unix groups. Its static initializer requires NativeCodeLoader.isNativeCodeLoaded(); if the Hadoop native library (libhadoop.so) is not loaded it throws RuntimeException and class initialization fails. Because the class is instantiated reflectively from hadoop.security.group.mapping, the visible symptom in logs is usually an ExceptionInInitializerError / 'Failed to load new group mapping class' followed by fallback or startup failure.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/JniBasedUnixGroupsMapping.java:49

import org.apache.hadoop.util.NativeCodeLoader;
import org.slf4j.Logger;
import org.slf4j.LoggerFactory;

/**
 * A JNI-based implementation of {@link GroupMappingServiceProvider} 
 * that invokes libC calls to get the group
 * memberships of a given user.
 */
@InterfaceAudience.LimitedPrivate({"HDFS", "MapReduce"})
@InterfaceStability.Evolving
public class JniBasedUnixGroupsMapping implements GroupMappingServiceProvider {
  
  private static final Logger LOG =
      LoggerFactory.getLogger(JniBasedUnixGroupsMapping.class);

  static {
    if (!NativeCodeLoader.isNativeCodeLoaded()) {
      throw new RuntimeException("Bailing out since native library couldn't " +
        "be loaded");
    }
    anchorNative();
    LOG.debug("Using JniBasedUnixGroupsMapping for Group resolution");
  }

  /**
   * Set up our JNI resources.
   *
   * @throws                 RuntimeException if setup fails.
   */
  native static void anchorNative();

  /**
   * Get the set of groups associated with a user.
   *
   * @param username           The user name
   *

View on GitHub (pinned to 2add963021)

Solutions

Check the startup log for the actual native-load failure ('Unable to load native-hadoop library') and fix the root cause: correct java.library.path/LD_LIBRARY_PATH to $HADOOP_HOME/lib/native
Compile or download the native library matching your platform (mvn package -Pnative -DskipTests in hadoop-common, or distro-specific package)
Switch hadoop.security.group.mapping to org.apache.hadoop.security.ShellBasedUnixGroupsMapping (pure Java, runs id -Gn) when natives are unavailable
Verify glibc/architecture compatibility (file libhadoop.so, ldd output) and that HADOOP_OPTS carries the library path into daemon JVMs

Example fix

# before (core-site.xml)
<property><name>hadoop.security.group.mapping</name>
  <value>org.apache.hadoop.security.JniBasedUnixGroupsMapping</value></property>

# after: fallback-safe mapping without natives
<property><name>hadoop.security.group.mapping</name>
  <value>org.apache.hadoop.security.JniBasedUnixGroupsMappingWithFallback</value></property>
# or pure-shell
<property><name>hadoop.security.group.mapping</name>
  <value>org.apache.hadoop.security.ShellBasedUnixGroupsMapping</value></property>

# and ensure natives are found
export HADOOP_OPTS="$HADOOP_OPTS -Djava.library.path=$HADOOP_HOME/lib/native"

Defensive patterns

Strategy: fallback

Validate before calling

if (conf.get("hadoop.security.group.mapping", "")
        .contains("JniBasedUnixGroupsMapping")
    && !NativeCodeLoader.isNativeCodeLoaded()) {
  LOG.warn("Native library not loaded; JNI group mapping will fail. "
      + "Falling back to ShellBasedUnixGroupsMapping.");
  conf.set("hadoop.security.group.mapping",
      "org.apache.hadoop.security.ShellBasedUnixGroupsMapping");
}

Try / catch

try {
  Class<?> c = Class.forName("org.apache.hadoop.security.JniBasedUnixGroupsMapping");
  GroupMappingServiceProvider m = (GroupMappingServiceProvider) c.newInstance();
} catch (ExceptionInInitializerError | NoClassDefFoundError e) {
  // static init failed: natives missing — switch mapping implementation
  LOG.warn("JNI group mapping unavailable: {}", e.toString());
}

Prevention

Verify natives at rollout: run `hadoop checknative` on every node
Use ...WithFallback mapping classes so resolution degrades to shell instead of failing
Propagate java.library.path/LD_LIBRARY_PATH into daemon environments explicitly

When it happens

Trigger: hadoop.security.group.mapping=org.apache.hadoop.security.JniBasedUnixGroupsMapping (often via the default JniBasedUnixGroupsMappingWithFallback going wrong) while libhadoop.so is absent for the platform, LD_LIBRARY_PATH/HADOOP_HOME/lib/native lacks it, glibc is incompatible, or the JVM is 64-bit with only 32-bit natives.

Common situations: Running on exotic/archaic OSes without prebuilt natives; LD_LIBRARY_PATH not propagated to daemon environments; HADOOP_OPTS missing -Djava.library.path; macOS/arm or Alpine musl where bundled .so won't load; downgrading/upgrading Hadoop leaving stale native dirs.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/17917afca8f8507d. Report an issue: GitHub.