apache/hadoop · error · RuntimeException

Bailing out since native library couldn't be loaded

Error message

Bailing out since native library couldn't be loaded

What it means

JniBasedUnixGroupsNetgroupMapping (netgroup resolution via getent-style JNI) has the same static guard as its parent: it requires NativeCodeLoader.isNativeCodeLoaded() or it throws RuntimeException during class initialization. It extends JniBasedUnixGroupsMapping and additionally declares getUsersForNetgroupJNI, so both the base class initializer and its own must see natives loaded. Typical symptom: ExceptionInInitializerError when Hadoop first resolves groups/netgroups after you configured this mapping.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/security/JniBasedUnixGroupsNetgroupMapping.java:53

/**
 * A JNI-based implementation of {@link GroupMappingServiceProvider} 
 * that invokes libC calls to get the group
 * memberships of a given user.
 */
@InterfaceAudience.LimitedPrivate({"HDFS", "MapReduce"})
@InterfaceStability.Evolving
public class JniBasedUnixGroupsNetgroupMapping
  extends JniBasedUnixGroupsMapping {
  
  private static final Logger LOG = LoggerFactory.getLogger(
    JniBasedUnixGroupsNetgroupMapping.class);

  native String[] getUsersForNetgroupJNI(String group);

  static {
    if (!NativeCodeLoader.isNativeCodeLoaded()) {
      throw new RuntimeException("Bailing out since native library couldn't " +
        "be loaded");
    }
    LOG.debug("Using JniBasedUnixGroupsNetgroupMapping for Netgroup resolution");
  }

  /**
   * Gets unix groups and netgroups for the user.
   *
   * It gets all unix groups as returned by id -Gn but it
   * only returns netgroups that are used in ACLs (there is
   * no way to get all netgroups for a given user, see
   * documentation for getent netgroup)
   */
  @Override
  public List<String> getGroups(String user) throws IOException {
    // parent gets unix groups
    List<String> groups = new LinkedList<String>(super.getGroups(user));
    NetgroupCache.getNetgroups(user, groups);

View on GitHub (pinned to 2add963021)

Solutions

1. Fix native library loading: point java.library.path/LD_LIBRARY_PATH at $HADOOP_HOME/lib/native and confirm 'file libhadoop.so' matches your arch
2. Build platform-native artifacts (hadoop-common mvn -Pnative) or install your distribution's hadoop-native package on every node
3. If netgroups are required but natives impossible, use org.apache.hadoop.security.ShellBasedUnixGroupsNetgroupMapping (shells out to getent netgroup)
4. Prefer JniBasedUnixGroupsNetgroupMappingWithFallback so resolution degrades to shell instead of failing hard

Example fix

# before (core-site.xml)
<property><name>hadoop.security.group.mapping</name>
  <value>org.apache.hadoop.security.JniBasedUnixGroupsNetgroupMapping</value></property>

# after
<property><name>hadoop.security.group.mapping</name>
  <value>org.apache.hadoop.security.JniBasedUnixGroupsNetgroupMappingWithFallback</value></property>

if (conf.get("hadoop.security.group.mapping", "")
        .contains("Netgroup")
    && !NativeCodeLoader.isNativeCodeLoaded()) {
  LOG.warn("Natives missing; netgroup JNI mapping unavailable — "
      + "using ShellBasedUnixGroupsNetgroupMapping");
  conf.set("hadoop.security.group.mapping",
      "org.apache.hadoop.security.ShellBasedUnixGroupsNetgroupMapping");
}

try {
  Class<?> c = Class.forName(
    "org.apache.hadoop.security.JniBasedUnixGroupsNetgroupMapping");
  return (GroupMappingServiceProvider) c.newInstance();
} catch (ExceptionInInitializerError | NoClassDefFoundError e) {
  // natives not loaded; fall back to shell-based netgroup mapping
  return new ShellBasedUnixGroupsNetgroupMapping();
}

When it happens

Trigger: hadoop.security.group.mapping=org.apache.hadoop.security.JniBasedUnixGroupsNetgroupMapping (or the ...WithFallback variant escalating to it) when libhadoop.so is not loadable — missing native dir in java.library.path, wrong architecture/glibc, or natives never built for the platform.

Common situations: HDFS ACLs or permissions using netgroups on hosts where natives are stripped (minimal container images); deployments that copied config from a native-enabled cluster onto JVM-only nodes; OS upgrades breaking glibc compatibility of the bundled .so.

Related errors

• Bailing out since native library couldn't be loaded
• Secure IO is not possible without native code extensions.
• lz4-java library is not available: Lz4Compressor has not bee
• lz4-java library is not available: Lz4Decompressor has not b
• Configuration hadoop.user.group.static.mapping.overrides is

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/84f3b3c9872aa6a9. Report an issue: GitHub.