{"record":{"id":"fda8c89a084d7ec2","repo":"apache/flink","slug":"s3-file-system-cannot-recover-recoverable-for-othe","errorCode":null,"errorMessage":"S3 File System cannot recover recoverable for other file system: {}","messagePattern":"S3 File System cannot recover recoverable for other file system: (.+?)","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"flink-filesystems/flink-s3-fs-base/src/main/java/org/apache/flink/fs/s3/common/writer/S3RecoverableWriter.java","lineNumber":139,"sourceCode":"\n    @Override\n    @SuppressWarnings({\"rawtypes\", \"unchecked\"})\n    public SimpleVersionedSerializer<ResumeRecoverable> getResumeRecoverableSerializer() {\n        return (SimpleVersionedSerializer) S3RecoverableSerializer.INSTANCE;\n    }\n\n    @Override\n    public boolean supportsResume() {\n        return true;\n    }\n\n    // --------------------------- Utils ---------------------------\n\n    private static S3Recoverable castToS3Recoverable(CommitRecoverable recoverable) {\n        if (recoverable instanceof S3Recoverable) {\n            return (S3Recoverable) recoverable;\n        }\n        throw new IllegalArgumentException(\n                \"S3 File System cannot recover recoverable for other file system: \" + recoverable);\n    }\n\n    // --------------------------- Static Constructor ---------------------------\n\n    public static S3RecoverableWriter writer(\n            final FileSystem fs,\n            final FunctionWithException<File, RefCountedFileWithStream, IOException>\n                    tempFileCreator,\n            final S3AccessHelper s3AccessHelper,\n            final Executor uploadThreadPool,\n            final long userDefinedMinPartSize,\n            final int maxConcurrentUploadsPerStream) {\n\n        checkArgument(userDefinedMinPartSize >= S3_MULTIPART_MIN_PART_SIZE);\n\n        final S3RecoverableMultipartUploadFactory uploadFactory =\n                new S3RecoverableMultipartUploadFactory(","sourceCodeStart":121,"sourceCodeEnd":157,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-filesystems/flink-s3-fs-base/src/main/java/org/apache/flink/fs/s3/common/writer/S3RecoverableWriter.java#L121-L157","documentation":"S3RecoverableWriter.castToS3Recoverable throws IllegalArgumentException when a CommitRecoverable that is not an S3Recoverable instance is passed to the writer's recover, commit, or resume operations. The S3 writer can only finish uploads that its own S3RecoverableWriter started, because the recoverable carries S3-specific data (uploadId, part ETags).","triggerScenarios":"Calling S3RecoverableWriter.recoverForCommit / commit / resume with a CommitRecoverable produced by a different filesystem's RecoverableWriter (e.g. local filesystem, HDFS, or another object store), or with a mock/forged CommitRecoverable object.","commonSituations":"Job graph rewired to a different output filesystem between the write phase and the commit phase (e.g. scheme changed from s3:// to file:// in config), a sink that stores GenericWriterProperties instead of concrete recoverables, or unit tests passing dummy CommitRecoverable implementations.","solutions":["Ensure the filesystem scheme used to create the RecoverableWriter is identical to the one used when committing; both phases must resolve to the same S3 filesystem.","Check that the sink serialized the concrete S3Recoverable (via S3RecoverableSerializer) and not some other CommitRecoverable type.","In tests, construct S3Recoverable instances (or use S3RecoverableSerializer round-trips) instead of anonymous CommitRecoverable stubs.","If you need to support multiple filesystems, branch on recoverable type before choosing the writer."],"exampleFix":"// before\nRecoverableWriter s3Writer = s3Fs.createRecoverableWriter();\ns3Writer.recoverForCommit(recoverableFromLocalFs); // IllegalArgumentException\n\n// after\nif (recoverable instanceof S3Recoverable) {\n    s3Writer.recoverForCommit(recoverable);\n} else {\n    RecoverableWriter correctWriter = originFs.createRecoverableWriter();\n    correctWriter.recoverForCommit(recoverable);\n}","handlingStrategy":"type-guard","validationCode":null,"typeGuard":"private static S3Recoverable asS3Recoverable(CommitRecoverable r) {\n    if (r instanceof S3Recoverable) {\n        return (S3Recoverable) r;\n    }\n    return null; // caller routes to the matching writer instead of failing\n}","tryCatchPattern":"try {\n    s3Writer.recoverForCommit(recoverable);\n} catch (IllegalArgumentException e) {\n    // recoverable belongs to another filesystem: re-acquire the correct\n    // RecoverableWriter from the originating FileSystem and retry there\n}","preventionTips":["Keep the output filesystem scheme identical between the write and commit phases of a job.","Store the concrete recoverable type alongside the serialized bytes when persisting across restarts.","In tests, always use real S3Recoverable instances or serializer round-trips."],"tags":["s3","api-misuse","recoverable-writer","type-check"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}