apache/hadoop · error · PathIOException

FileSystem is closed!

Error message

FileSystem is closed!

What it means

Public S3AFileSystem entry points call checkNotClosed(), which throws PathIOException(uri, 'FileSystem is closed!') (E_FS_CLOSED) once the volatile isClosed flag is set by close(). The instance cannot be revived; this is a use-after-close programming error, not a transient condition - no retry will help.

Source

Thrown at hadoop-tools/hadoop-aws/src/main/java/org/apache/hadoop/fs/s3a/S3AFileSystem.java:4428

      });
    } catch (IOException e) {
      // failure during shutdown.
      // this should only be from the signature of trackDurationAndSpan().
      LOG.warn("Failure during service shutdown", e);
    }
    // and once this duration has been tracked, close the statistics
    // other services are shutdown.
    cleanupWithLogger(LOG, instrumentation);
  }

  /**
   * Verify that the filesystem has not been closed. Non blocking; this gives
   * the last state of the volatile {@link #closed} field.
   * @throws PathIOException if the FS is closed.
   */
  private void checkNotClosed() throws PathIOException {
    if (isClosed) {
      throw new PathIOException(uri.toString(), E_FS_CLOSED);
    }
  }

  /**
   * Get the delegation token support for this filesystem;
   * not null iff delegation support is enabled.
   * @return the token support, or an empty option.
   */
  @VisibleForTesting
  public Optional<S3ADelegationTokens> getDelegationTokens() {
    return delegationTokens;
  }

  /**
   * Return a service name iff delegation tokens are enabled and the
   * token binding is issuing delegation tokens.
   * @return the canonical service name or null
   */

View on GitHub (pinned to 2add963021)

Solutions

1. Never close FileSystems obtained from FileSystem.get() - they are cache-shared; only close instances you created with newInstance()
2. Scope all usage of a FileSystem inside the block that owns it
3. On this error, discard the reference and obtain a fresh instance with FileSystem.newInstance(uri, conf)

Example fix

// before: closes the shared cached instance; later users fail
try (FileSystem fs = FileSystem.get(uri, conf)) {
  fs.rename(src, dst);
}

// after: cached instance is not closed; ownership stays with the cache
FileSystem fs = FileSystem.get(uri, conf);
fs.rename(src, dst);
// or, for explicit lifecycle control:
try (FileSystem own = FileSystem.newInstance(uri, conf)) {
  own.rename(src, dst);
}

// Single-owner wrapper: one place creates and closes the FS
try (FileSystem own = FileSystem.newInstance(uri, conf)) {
  own.rename(src, dst);
}

static boolean isFsClosed(PathIOException e) {
  return "FileSystem is closed!".equals(e.getMessage());
}

Catch PathIOException, test for the closed message (E_FS_CLOSED), then rebuild the client (FileSystem.newInstance) and fail or retry the operation once with the fresh instance - the old reference is permanently dead.

When it happens

Trigger: Any FS operation invoked after fs.close() returned; a try-with-resources block on a shared/cached FileSystem closing it while other threads still use it; test teardown closing an instance that later assertions touch.

Common situations: Helper methods closing FileSystems they obtained via FileSystem.get() (which returns a shared cached instance); shutdown hooks (FileSystem.closeAllForUGI) racing in-flight work; Spark executors continuing to use a FileSystem after the context stopped.

Related errors

• Credentials requested after provider list was closed
• Stream is closed!
• Stream closed
• {}: FileSystem is closed!
• Stream is closed!

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/ee0ec9e867fd800e. Report an issue: GitHub.