apache/hadoop · error · IOException

NativeIO was not loaded

Error message

NativeIO was not loaded

What it means

NativeIO.assertCodeLoaded() throws IOException("NativeIO was not loaded") when the JNI-based native IO layer is unavailable: either NativeCodeLoader reports no native libhadoop loaded, or NativeIO's own initializer failed. Every JNI-dependent NativeIO entry point (mlock, fstat, getStat, chmod wrappers, renameTo0) funnels through this guard, so any of them can surface this error.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/nativeio/NativeIO.java:369

        } catch (Throwable t) {
          // This can happen if the user has an older version of libhadoop.so
          // installed - in this case we can continue without native IO
          // after warning
          PerformanceAdvisory.LOG.debug("Unable to initialize NativeIO libraries", t);
        }
      }
    }

    /**
     * @return Return true if the JNI-based native IO extensions are available.
     */
    public static boolean isAvailable() {
      return NativeCodeLoader.isNativeCodeLoaded() && nativeLoaded;
    }

    private static void assertCodeLoaded() throws IOException {
      if (!isAvailable()) {
        throw new IOException("NativeIO was not loaded");
      }
    }

    /**
     * Wrapper around open(2) .
     * @param path input path.
     * @param flags input flags.
     * @param mode input mode.
     * @return FileDescriptor.
     * @throws IOException raised on errors performing I/O.
     */
    public static native FileDescriptor open(String path, int flags, int mode) throws IOException;
    /** Wrapper around fstat(2) */
    private static native Stat fstat(FileDescriptor fd) throws IOException;
    /** Wrapper around stat(2). */
    private static native Stat stat(String path) throws IOException;

    /** Native chmod implementation. On UNIX, it is a wrapper around chmod(2) */

View on GitHub (pinned to 2add963021)

Solutions

1. Check availability up front with NativeIO.isAvailable() and read the NativeCodeLoader startup banner — it logs exactly which library failed and why.
2. Point the JVM at the natives: -Djava.library.path=$HADOOP_HOME/lib/native (or export LD_LIBRARY_PATH) and verify libhadoop.so exists for your OS/arch with ldd.
3. If natives are missing or incompatible, install/rebuild the correct package (distro package or mvn native compile) matching arch and glibc.
4. If native IO is optional in your path, branch to a pure-Java implementation (java.nio.file) when isAvailable() is false.

Example fix

// before
NativeIO.chmod(path, 0644); // IOException: NativeIO was not loaded

// after
if (NativeIO.isAvailable()) {
  NativeIO.chmod(path, 0644);
} else {
  Files.setPosixFilePermissions(Paths.get(path),
      PosixFilePermissions.fromString("rw-r--r--"));
}

if (!NativeIO.isAvailable()) {
  // skip native-dependent calls; route to a java.nio.file implementation
}

try {
  NativeIO.getStat(path);
} catch (IOException e) {
  if ("NativeIO was not loaded".equals(e.getMessage())) {
    // fall back: Files.readAttributes(Paths.get(path), ...)
  } else { throw e; }
}

When it happens

Trigger: Calling a JNI-dependent NativeIO method on a JVM where libhadoop.so/hadoop.dll was never loaded or failed to load: java.library.path not pointing at $HADOOP_HOME/lib/native, missing or wrong-arch .so, glibc incompatibility, unsupported platform, or a broken Hadoop distribution layout.

Common situations: LD_LIBRARY_PATH/-Djava.library.path unset in custom launch scripts; running on platforms without prebuilt natives (some ARM/macOS setups); Windows without hadoop.dll or the VC++ runtime; containers with a stripped-down Hadoop install.

Related errors

• Bad format: {}
• src.toString() + ": No such file or directory"
• Range starts beyond the file length ({l}): {last}
• getXAttrs on path `{}' is not within a mount point
• listXAttrs on path `{}' is not within a mount point

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/97399942c36cf6ff. Report an issue: GitHub.