{"record":{"id":"70b39dfb0a0faf4d","repo":"apache/flink","slug":"failed-to-complete-multipart-upload-for-key","errorCode":null,"errorMessage":"Failed to complete multipart upload for key: {}","messagePattern":"Failed to complete multipart upload for key: (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"flink-filesystems/flink-s3-fs-native/src/main/java/org/apache/flink/fs/s3native/writer/NativeS3ObjectOperations.java","lineNumber":336,"sourceCode":"                    CompleteMultipartUploadRequest.builder()\n                            .bucket(bucketName)\n                            .key(key)\n                            .uploadId(uploadId)\n                            .multipartUpload(\n                                    CompletedMultipartUpload.builder()\n                                            .parts(completedParts)\n                                            .build())\n                            .build();\n\n            CompleteMultipartUploadResponse response = s3Client.completeMultipartUpload(request);\n            return new CompleteMultipartUploadResult(\n                    bucketName, key, response.eTag(), response.location());\n        } catch (NoSuchUploadException e) {\n            try {\n                ObjectMetadata metadata = getObjectMetadata(key);\n                return new CompleteMultipartUploadResult(bucketName, key, metadata.getETag(), null);\n            } catch (IOException checkEx) {\n                throw new IOException(\"Failed to complete multipart upload for key: \" + key, e);\n            }\n        } catch (S3Exception e) {\n            throw new IOException(\"Failed to complete multipart upload for key: \" + key, e);\n        }\n    }\n\n    public void abortMultiPartUpload(String key, String uploadId) throws IOException {\n        try {\n            AbortMultipartUploadRequest request =\n                    AbortMultipartUploadRequest.builder()\n                            .bucket(bucketName)\n                            .key(key)\n                            .uploadId(uploadId)\n                            .build();\n\n            s3Client.abortMultipartUpload(request);\n        } catch (S3Exception e) {\n            throw new IOException(","sourceCodeStart":318,"sourceCodeEnd":354,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-filesystems/flink-s3-fs-native/src/main/java/org/apache/flink/fs/s3native/writer/NativeS3ObjectOperations.java#L318-L354","documentation":"completeMultipartUpload() catches NoSuchUploadException and, as a recovery heuristic, checks whether the object already exists (getObjectMetadata) — the upload may have been completed by a previous attempt before a failover. If that existence check ALSO fails (object absent), this IOException is thrown wrapping the ORIGINAL NoSuchUploadException: the upload is gone and there is no committed object, so the write is genuinely lost from this handle's perspective. This is the failure that indicates an aborted/expired upload rather than a completed one.","triggerScenarios":"Multipart upload aborted by another attempt (duplicate committer after recovery), aborted by an S3 lifecycle rule (AbortIncompleteMultipartUpload), uploadId expired, or the complete succeeded on a previous attempt but the object was since deleted — then getObjectMetadata throws and this error surfaces with the NoSuchUpload cause.","commonSituations":"Two committers racing after task failover (one aborts, one completes); bucket lifecycle rules aggressively cleaning incomplete uploads (e.g. 1 day) while jobs hold recoverable state longer; user/scripts deleting objects mid-recovery; cross-run dedup logic aborting 'stale' uploads it should not have.","solutions":["Ensure exactly one committer per upload: rely on Flink's exactly-once committer semantics / FileSink committer rather than calling commit() from multiple recovered attempts.","Review bucket lifecycle AbortIncompleteMultipartUpload days vs your longest expected recovery window; raise it above your max checkpoint/savepoint restore delay.","Examine the cause: NoSuchUpload + missing object means this write must be redone — re-run from the last good checkpoint (data since the upload start is not recoverable from this handle).","Audit external automation that aborts multipart uploads or deletes objects in the target prefix."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// Before committing a recovered upload, confirm it is still live\ntry {\n    s3Client.listMultipartUploads(b -> b.bucket(bucket).prefix(key));\n    // if uploadId absent from listing AND object absent via headObject -> upload lost: plan re-write\n} catch (S3Exception e) { /* surface config/permission problem early */ }","typeGuard":null,"tryCatchPattern":"catch (IOException e) { if (e.getCause() instanceof NoSuchUploadException) { // upload aborted/expired and object absent -> data must be re-written from last checkpoint; never retry this commit } else throw e; }","preventionTips":["Set bucket lifecycle AbortIncompleteMultipartUpload above your worst-case recovery time.","Guarantee a single committer per upload (Flink committable semantics) so attempts never abort each other.","Keep checkpoint intervals short enough that recovered uploads commit well inside any lifecycle window."],"tags":["s3","multipart-upload","recovery","lifecycle-rules","commit","flink"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}