{"record":{"id":"0a3d9030fe2d364d","repo":"apache/flink","slug":"recoverable-writer-not-available","errorCode":null,"errorMessage":"Recoverable writer not available","messagePattern":"Recoverable writer not available","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"flink-filesystems/flink-s3-fs-native/src/main/java/org/apache/flink/fs/s3native/NativeS3FileSystem.java","lineNumber":537,"sourceCode":"\n    @Override\n    public void copyFiles(\n            List<CopyRequest> requests,\n            org.apache.flink.core.fs.ICloseableRegistry closeableRegistry)\n            throws IOException {\n        checkNotClosed();\n        if (bulkCopyHelper == null) {\n            throw new UnsupportedOperationException(\n                    \"Bulk copy not enabled. Set s3.bulk-copy.enabled=true\");\n        }\n        bulkCopyHelper.copyFiles(requests, closeableRegistry);\n    }\n\n    @Override\n    public RecoverableWriter createRecoverableWriter() throws IOException {\n        checkNotClosed();\n        if (s3AccessHelper == null) {\n            throw new UnsupportedOperationException(\"Recoverable writer not available\");\n        }\n        return NativeS3RecoverableWriter.writer(\n                s3AccessHelper, localTmpDir, s3uploadPartSize, maxConcurrentUploadsPerStream);\n    }\n\n    @Override\n    public CompletableFuture<Void> closeAsync() {\n        if (!closed.compareAndSet(false, true)) {\n            return CompletableFuture.completedFuture(null);\n        }\n\n        LOG.info(\"Starting async close of Native S3 FileSystem for bucket: {}\", bucketName);\n        CompletableFuture<Void> closeFuture =\n                CompletableFuture.runAsync(\n                                () ->\n                                        LOG.info(\n                                                \"Native S3 FileSystem closed for bucket: {}\",\n                                                bucketName))","sourceCodeStart":519,"sourceCodeEnd":555,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-filesystems/flink-s3-fs-native/src/main/java/org/apache/flink/fs/s3native/NativeS3FileSystem.java#L519-L555","documentation":"NativeS3FileSystem.createRecoverableWriter throws UnsupportedOperationException when s3AccessHelper is null. The access helper (object operations for multipart uploads) is optional in this filesystem; without it there is no way to implement recoverable (resumable, exactly-once) writes, so the method fails fast rather than returning a broken writer.","triggerScenarios":"Calling createRecoverableWriter() on a NativeS3FileSystem instance constructed without an S3AccessHelper — e.g. a filesystem built for read-only or bulk-copy-only usage, or a factory configuration path that skips upload support.","commonSituations":"Exactly-once sinks (files sink with exactly-once commit, recoverable writers) configured against a filesystem variant that was constructed without upload capabilities; misconfigured factory wiring in embedded/tests setups.","solutions":["Construct/obtain the filesystem through the standard factory so the S3AccessHelper is created (ensure credentials and upload settings are configured).","If you only need plain writes, use create(...) streams instead of a RecoverableWriter.","For sinks requiring exactly-once file commits on S3, verify the filesystem plugin (flink-s3-fs-native shaded jar) is properly installed under plugins/ or lib/."],"exampleFix":"// before\nRecoverableWriter w = readOnlyS3Fs.createRecoverableWriter(); // throws\n\n// after\n// obtain the filesystem via the factory with full configuration:\nFileSystem fs = FileSystem.get(new Path(\"s3://bucket/\").toUri());\nRecoverableWriter w = fs.createRecoverableWriter();","handlingStrategy":"type-guard","validationCode":null,"typeGuard":"static boolean canCreateRecoverableWriter(FileSystem fs) {\n    // NativeS3FileSystem only supports it when constructed with the S3 access helper;\n    // probe cheaply and catch UnsupportedOperationException on first call at startup\n    try {\n        fs.createRecoverableWriter();\n        return true;\n    } catch (UnsupportedOperationException e) {\n        return false;\n    }\n}","tryCatchPattern":"try {\n    RecoverableWriter writer = fs.createRecoverableWriter();\n} catch (UnsupportedOperationException e) {\n    // filesystem built without upload support: use plain create() streams or\n    // re-obtain the filesystem via the standard factory with full S3 config\n}","preventionTips":["Acquire S3 filesystems via FileSystem.get with fully configured factories so the access helper is present.","Probe createRecoverableWriter() once at sink initialization, not per record.","Install the proper S3 filesystem plugin (flink-s3-fs-native) for exactly-once file sinks."],"tags":["s3","recoverable-writer","unsupported-operation","configuration"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}