apache/hadoop · critical · RuntimeException

Couldn't obtain an instance of RawLocalFileSystem.

Error message

Couldn't obtain an instance of RawLocalFileSystem.

What it means

Thrown from SecureIOUtils' static initializer when FileSystem.getLocal(new Configuration()) throws while pre-caching a RawLocalFileSystem. The instance is cached eagerly because secure I/O sometimes runs in shutdown hooks where filesystem lookup would fail. Like the sibling security error, it surfaces as ExceptionInInitializerError wrapping this RuntimeException on first use of the class.

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SecureIOUtils.java:80

   *
   * If security is enabled but the support code is unavailable, throws a
   * RuntimeException since we don't want to run insecurely.
   */
  static {
    boolean shouldBeSecure = UserGroupInformation.isSecurityEnabled();
    boolean canBeSecure = NativeIO.isAvailable();

    if (!canBeSecure && shouldBeSecure) {
      throw new RuntimeException(
        "Secure IO is not possible without native code extensions.");
    }

    // Pre-cache an instance of the raw FileSystem since we sometimes
    // do secure IO in a shutdown hook, where this call could fail.
    try {
      rawFilesystem = FileSystem.getLocal(new Configuration()).getRaw();
    } catch (IOException ie) {
      throw new RuntimeException(
      "Couldn't obtain an instance of RawLocalFileSystem.");
    }

    // SecureIO just skips security checks in the case that security is
    // disabled
    skipSecurity = !canBeSecure;
  }

  private final static boolean skipSecurity;
  private final static FileSystem rawFilesystem;

  /**
   * @return Open the given File for random read access, verifying the expected user/
   * group constraints if security is enabled.
   * 
   * Note that this function provides no additional security checks if hadoop
   * security is disabled, since doing the checks would be too expensive when
   * native libraries are not available.

View on GitHub (pinned to 2add963021)

Solutions

1. Reproduce the root cause in a scratch program: FileSystem.getLocal(new Configuration()).getRaw() and read the underlying IOException stack
2. Verify core-site.xml on the classpath is valid XML and remove stale fs.file.impl / fs.raw-file-system.impl overrides
3. Ensure hadoop-common jar and etc/hadoop config dir are both on the client classpath
4. If your code closes the FileSystem cache, do it only after all SecureIOUtils work is done

Example fix

// before: bad fs.file.impl in core-site.xml -> ExceptionInInitializerError at class load
SecureIOUtils.createForWrite(f, 0644);

// after: startup smoke test that surfaces the real IOException
FileSystem local = FileSystem.getLocal(new Configuration()).getRaw(); // fails loudly at deploy time
LOG.info("local fs ok: {}", local.getUri());

// startup smoke test: if this throws, SecureIOUtils class init will too
FileSystem raw = FileSystem.getLocal(new Configuration()).getRaw();
LOG.info("RawLocalFileSystem ok: {}", raw.getUri());

try {
  SecureIOUtils.openForRead(f, owner);
} catch (ExceptionInInitializerError e) {
  Throwable cause = e.getCause(); // RuntimeException with the real message
  if (cause != null && cause.getMessage().contains("RawLocalFileSystem")) {
    // local filesystem config problem, not the native-library problem
  }
  throw e;
}

When it happens

Trigger: Class initialization in a JVM whose configuration makes local filesystem creation throw IOException: fs.file.impl / fs.raw-file-system.impl set to a class that is not on the classpath or fails to construct, core-site.xml unreadable or syntactically broken, or the FileSystem cache already closed (FileSystem.closeAll()) before first use.

Common situations: Custom fs.file.impl overrides left in core-site.xml after a jar upgrade; shaded/relocated fat jars breaking FileSystem service loading; embedded apps that call FileSystem.closeAllForUGI and later touch SecureIOUtils; containers missing HADOOP_CONF_DIR so Configuration loads no valid local fs binding.

Related errors

• Uri without authority: {uri}
• Wrong FS: {path}, expected: {this.getUri()}
• dir + " is not a directory!"
• Error in instantiating YarnClient
• No more entry in " + f

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/09f7afcbd2896882. Report an issue: GitHub.