apache/hadoop · critical · RuntimeException

Secure IO is not possible without native code extensions.

Error message

Secure IO is not possible without native code extensions.

What it means

Thrown from SecureIOUtils' static initializer when Hadoop security (Kerberos) is enabled but NativeIO.isAvailable() is false, i.e. libhadoop.so could not be loaded. Because it fires during class initialization, callers normally see an ExceptionInInitializerError wrapping this RuntimeException the first time any SecureIOUtils method is touched. The class refuses to operate this way because owner checks and secure file creation cannot be done safely without native code.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SecureIOUtils.java:71

 * <br>
 */
public class SecureIOUtils {

  /**
   * Ensure that we are set up to run with the appropriate native support code.
   * If security is disabled, and the support code is unavailable, this class
   * still tries its best to be secure, but is vulnerable to some race condition
   * attacks.
   *
   * If security is enabled but the support code is unavailable, throws a
   * RuntimeException since we don't want to run insecurely.
   */
  static {
    boolean shouldBeSecure = UserGroupInformation.isSecurityEnabled();
    boolean canBeSecure = NativeIO.isAvailable();

    if (!canBeSecure && shouldBeSecure) {
      throw new RuntimeException(
        "Secure IO is not possible without native code extensions.");
    }

    // Pre-cache an instance of the raw FileSystem since we sometimes
    // do secure IO in a shutdown hook, where this call could fail.
    try {
      rawFilesystem = FileSystem.getLocal(new Configuration()).getRaw();
    } catch (IOException ie) {
      throw new RuntimeException(
      "Couldn't obtain an instance of RawLocalFileSystem.");
    }

    // SecureIO just skips security checks in the case that security is
    // disabled
    skipSecurity = !canBeSecure;
  }

  private final static boolean skipSecurity;

View on GitHub (pinned to 2add963021)

Solutions

1. Run 'hadoop checknative -a' on the failing node and fix whatever it reports as missing/unloadable
2. Point java.library.path (or LD_LIBRARY_PATH) at the directory containing libhadoop.so and restart the JVM
3. Rebuild native code for the target platform (mvn package -Pnative -Drequire.snappy ... in hadoop-common) and deploy lib/native
4. If native code truly cannot be provided, set hadoop.security.authentication=simple and hadoop.security.authorization=false and restart all daemons/clients

Example fix

// before: Kerberos on, native lib absent -> ExceptionInInitializerError at first SecureIOUtils use
SecureIOUtils.openForRead(file, owner);

// after: gate usage on the same condition the static block checks
if (UserGroupInformation.isSecurityEnabled() && !NativeIO.isAvailable()) {
  throw new IllegalStateException(
      "SecureIOUtils requires native libraries when Kerberos is enabled; run 'hadoop checknative -a'");
}
SecureIOUtils.openForRead(file, owner);

// run before touching SecureIOUtils
if (UserGroupInformation.isSecurityEnabled() && !NativeIO.isAvailable()) {
  throw new IllegalStateException(
      "Kerberos enabled but native IO unavailable; run 'hadoop checknative -a' and set java.library.path");
}

try {
  SecureIOUtils.openForRead(f, owner);
} catch (ExceptionInInitializerError e) {
  // static init failed: native libs missing under Kerberos (or local fs config broken)
  throw new RuntimeException("SecureIOUtils init failed: " + e.getCause(), e);
}

When it happens

Trigger: Any first reference to SecureIOUtils (openForRead, createForWrite, etc.) on a JVM where UserGroupInformation.isSecurityEnabled() returns true and NativeIO.isAvailable() returns false: HADOOP native dir missing libhadoop.so for the platform, java.library.path/LD_LIBRARY_PATH not pointing at it, or the library failing to load (glibc/ABI mismatch).

Common situations: Enabling Kerberos on nodes whose native tarball was never installed or was built for another OS/arch; upgrading Hadoop without redeploying native libs; minimal Docker images that strip lib/native; test harnesses that flip hadoop.security.authentication to kerberos without shipping native code.

Related errors

• Server asks us to fall back to SIMPLE auth, but this client
• User {} is not authorized for protocol {}: {}
• Security is enabled but block access tokens (via dfs.block.a
• Access denied: dfs.http.policy is HTTPS_ONLY.
• No URI in deserialized Path

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/1c1e2f344eca46fb. Report an issue: GitHub.