{"record":{"id":"def9d98a8c82815f","repo":"apache/hadoop","slug":"filesystem-is-closed","errorCode":null,"errorMessage":"{}: FileSystem is closed!","messagePattern":"(.+?): FileSystem is closed!","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/sftp/SFTPFileSystem.java","lineNumber":731,"sourceCode":"      super.close();\n      if (closed.getAndSet(true)) {\n        return;\n      }\n    } finally {\n      if (connectionPool != null) {\n        connectionPool.shutdown();\n      }\n    }\n  }\n\n  /**\n   * Verify that the input stream is open. Non blocking; this gives\n   * the last state of the volatile {@link #closed} field.\n   * @throws IOException if the connection is closed.\n   */\n  private void checkNotClosed() throws IOException {\n    if (closed.get()) {\n      throw new IOException(uri + \": \" + E_FS_CLOSED);\n    }\n  }\n\n  @VisibleForTesting\n  SFTPConnectionPool getConnectionPool() {\n    return connectionPool;\n  }\n}\n","sourceCodeStart":713,"sourceCodeEnd":740,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/sftp/SFTPFileSystem.java#L713-L740","documentation":"SFTPFileSystem.connect() begins with checkNotClosed() (defined at SFTPFileSystem.java:729), so once close() has run (closed AtomicBoolean set, connection pool shut down), every channel-acquiring operation — open, create, rename, delete, listStatus, getFileStatus, mkdirs — throws IOException formatted as '<uri>: FileSystem is closed!'.","triggerScenarios":"Calling any SFTPFileSystem data operation after fs.close(); reusing a FileSystem instance obtained from the FileSystem cache after some other component closed that shared cached instance; invoking FileSystem.closeAll() in a cleanup hook while the sftp:// filesystem is still in use.","commonSituations":"One library calls close() on the cached instance and a later caller gets the same closed object; finally-blocks that close prematurely before a retry loop; test code that closes the filesystem between cases but keeps the reference.","solutions":["Obtain a fresh, uncached instance with FileSystem.newInstance(uri, conf) instead of reusing or re-getting the closed one.","Audit for premature close(): close sftp:// filesystems only at application shutdown, and give each instance a single owner responsible for closing.","If a framework may close shared cached instances, disable caching (fs.<scheme>.impl.disable.cache=true) or re-acquire the instance per task."],"exampleFix":"// before\nFileSystem fs = FileSystem.get(sftpUri, conf);\nfs.close();\nfs.open(path); // IOException: FileSystem is closed!\n\n// after\nFileSystem fs = FileSystem.get(sftpUri, conf);\n// ... all work with fs ...\nfs.close(); // last statement; re-acquire via newInstance() if needed later","handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  in = fs.open(path);\n} catch (IOException e) {\n  if (e.getMessage() != null && e.getMessage().contains(\"FileSystem is closed\")) {\n    fs = FileSystem.newInstance(uri, conf); // fresh, uncached instance\n    in = fs.open(path);\n  } else {\n    throw e;\n  }\n}","preventionTips":["Assign each sftp:// FileSystem a single owner who closes it at shutdown only.","Prefer FileSystem.newInstance(uri, conf) when several components might close instances.","Never call FileSystem.closeAll() while other code still holds cached filesystems."],"tags":["sftp","filesystem-closed","lifecycle","hadoop","filesystem-cache"],"backgroundTag":"resource-already-closed","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}