apache/hadoop · error · IOException

GoogleHadoopFileSystem has been closed or not initialized.

Error message

GoogleHadoopFileSystem has been closed or not initialized.

What it means

GoogleHadoopFileSystem.checkOpen guards every operation with the isClosed flag and throws IOException('GoogleHadoopFileSystem has been closed or not initialized.') when the instance was close()d (or never completed initialize()). The message covers both lifecycle errors because the same flag defends both.

Source

Thrown at hadoop-cloud-storage-project/hadoop-gcp/src/main/java/org/apache/hadoop/fs/gs/GoogleHadoopFileSystem.java:700

  @Override
  public String getCanonicalServiceName() {
    // TODO: Add delegation token support
    return null;
  }

  /**
   * Gets GCS FS instance.
   */
  GoogleCloudStorageFileSystem getGcsFs() {
    return gcsFs;
  }

  /**
   * Assert that the FileSystem has been initialized and not close()d.
   */
  private void checkOpen() throws IOException {
    if (isClosed) {
      throw new IOException("GoogleHadoopFileSystem has been closed or not initialized.");
    }
  }

  @Override
  public void close() throws IOException {
    LOG.trace("close()");
    if (isClosed) {
      return;
    }

    super.close();

    getGcsFs().close();

    this.isClosed = true;
  }

  @Override

View on GitHub (pinned to 2add963021)

Solutions

Re-acquire the FileSystem after close: FileSystem.get(uri, conf) returns/creates a usable instance (or use FileSystem.newInstance for a private one).
Never cache a FileSystem across lifecycle boundaries; derive it from path.getFileSystem(conf) at use time.
If constructing directly, always call initialize(uri, conf) before any operation.

Example fix

// before
FileSystem fs = FileSystem.get(gcsUri, conf);
fs.close();
fs.open(path); // IOException: has been closed or not initialized.

// after
FileSystem fs = FileSystem.get(gcsUri, conf); // fresh usable instance
fs.open(path);

Defensive patterns

Strategy: validation

Validate before calling

FileSystem fs = path.getFileSystem(conf); // healthy per-use acquisition
// or, after a close:
FileSystem fresh = FileSystem.newInstance(gcsRootUri, conf);
fresh.open(path);

Try / catch

catch IOException with message equals("GoogleHadoopFileSystem has been closed or not initialized.") - abandon the dead instance and re-acquire via FileSystem.get/newInstance; do not retry on the same object.

Prevention

Do not cache FileSystem instances across job/service lifecycles; obtain per use via path.getFileSystem(conf).
Never close a shared/cached FileSystem from cleanup code that other consumers still use.
When constructing directly (tests), always call initialize(uri, conf) first.

When it happens

Trigger: Calling any FileSystem method (open, getFileStatus, rename, ...) on an instance after close(); using a manually constructed GoogleHadoopFileSystem without calling initialize(uri, conf); a shared cached instance closed by one job while another still uses it.

Common situations: Static/cached FileSystem handles in long-lived services where one shutdown path closes the shared instance; test code constructing the FS directly; job cleanup racing the next job's operations.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/c5b033980fc7eba8. Report an issue: GitHub.