{"record":{"id":"f77b01cca46c32ea","repo":"apache/flink","slug":"filesystem-has-been-closed-for-bucket-operati","errorCode":null,"errorMessage":"FileSystem has been closed for bucket: {}. Operations are no longer permitted.","messagePattern":"FileSystem has been closed for bucket: (.+?)\\. Operations are no longer permitted\\.","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"flink-filesystems/flink-s3-fs-native/src/main/java/org/apache/flink/fs/s3native/NativeS3FileSystem.java","lineNumber":593,"sourceCode":"                                        LOG.error(\n                                                \"FileSystem close did not complete cleanly within {} for bucket: {}\",\n                                                fsCloseTimeout,\n                                                bucketName,\n                                                error);\n                                    }\n                                });\n        FutureUtils.assertNoException(closeFuture);\n        return closeFuture;\n    }\n\n    /**\n     * Verifies that the filesystem has not been closed.\n     *\n     * @throws IllegalStateException if the filesystem has been closed\n     */\n    private void checkNotClosed() {\n        if (closed.get()) {\n            throw new IllegalStateException(\n                    \"FileSystem has been closed for bucket: \"\n                            + bucketName\n                            + \". Operations are no longer permitted.\");\n        }\n    }\n}\n","sourceCodeStart":575,"sourceCodeEnd":600,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-filesystems/flink-s3-fs-native/src/main/java/org/apache/flink/fs/s3native/NativeS3FileSystem.java#L575-L600","documentation":"Every mutating operation on NativeS3FileSystem starts with checkNotClosed(), which throws IllegalStateException once the closed AtomicBoolean is set by closeAsync(). S3 filesystem instances are cached and shut down on configuration changes or JVM/TaskManager shutdown, and using a stale reference afterwards is a programming error.","triggerScenarios":"Holding a NativeS3FileSystem reference across a closeAsync() (e.g. after FileSystem cache invalidation due to config change, or TaskManager shutdown) and then calling open/create/delete/rename/copyFiles on it.","commonSituations":"Caching FileSystem instances in user code beyond job lifecycle; reconfiguring fs.s3.* options which invalidates and closes cached instances; tests that close filesystems but keep references; races during shutdown where a task still writes output.","solutions":["Do not cache FileSystem instances; always re-acquire via FileSystem.get(uri) (the UDF/sink should fetch per use or rely on Flink's cache) so you get a live instance.","Ensure no writes happen after closing output streams / after job termination; sequence your shutdown after all tasks finish.","If triggered by a config change mid-run, restart the affected jobs so they pick up fresh, open filesystem instances."],"exampleFix":"// before\nprivate static final FileSystem FS = FileSystem.get(\"s3://bucket\");\n// ... later, after cache invalidation:\nFS.create(path, WriteMode.OVERWRITE); // IllegalStateException\n\n// after\n// re-acquire each time; FileSystem.get returns the cached (or new) open instance\nFileSystem fs = FileSystem.get(new Path(\"s3://bucket/\").toUri());\nfs.create(path, WriteMode.OVERWRITE);","handlingStrategy":"try-catch","validationCode":"// always re-acquire instead of caching\nFileSystem fs = FileSystem.get(new Path(\"s3://my-bucket/\").toUri());\nfs.create(path, WriteMode.OVERWRITE); // cache hands out a live instance","typeGuard":null,"tryCatchPattern":"try {\n    fs.create(path, WriteMode.OVERWRITE);\n} catch (IllegalStateException e) {\n    if (e.getMessage() != null && e.getMessage().contains(\"closed\")) {\n        fs = FileSystem.get(new Path(\"s3://my-bucket/\").toUri()); // re-acquire live instance\n        fs.create(path, WriteMode.OVERWRITE);\n    } else {\n        throw e;\n    }\n}","preventionTips":["Never store FileSystem instances in static fields; use FileSystem.get each time and let Flink's cache manage lifecycle.","Finish all writes before closing output / job teardown to avoid shutdown races.","After changing fs.s3.* config, expect cached instances to be closed — restart affected jobs."],"tags":["s3","lifecycle","use-after-close","filesystem-cache"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}