apache/flink · warning · IOException

Failed to abort multipart upload for key: %s, uploadId: %s

Error message

Failed to abort multipart upload for key: %s, uploadId: %s

What it means

abortMultiPartUpload() calls S3 AbortMultipartUpload for a (bucket, key, uploadId). Any S3Exception — typically NoSuchUpload (already completed/aborted) or AccessDenied — is wrapped in this formatted IOException carrying both the key and uploadId. Aborting frees storage for uploaded parts; S3 charges for parts of incomplete uploads until abort or lifecycle expiry, so unhandled failures here have a cost dimension.

Source

Thrown at flink-filesystems/flink-s3-fs-native/src/main/java/org/apache/flink/fs/s3native/writer/NativeS3ObjectOperations.java:354

                throw new IOException("Failed to complete multipart upload for key: " + key, e);
            }
        } catch (S3Exception e) {
            throw new IOException("Failed to complete multipart upload for key: " + key, e);
        }
    }

    public void abortMultiPartUpload(String key, String uploadId) throws IOException {
        try {
            AbortMultipartUploadRequest request =
                    AbortMultipartUploadRequest.builder()
                            .bucket(bucketName)
                            .key(key)
                            .uploadId(uploadId)
                            .build();

            s3Client.abortMultipartUpload(request);
        } catch (S3Exception e) {
            throw new IOException(
                    String.format(
                            "Failed to abort multipart upload for key: %s, uploadId: %s",
                            key, uploadId),
                    e);
        }
    }

    /**
     * Deletes an object from S3.
     *
     * <p><b>Permission Considerations:</b> Note that S3 may return different error codes depending
     * on permissions:
     *
     * <ul>
     *   <li>With ListBucket permission: Returns 404 if object doesn't exist
     *   <li>Without ListBucket permission: Returns 403 (Access Denied) even for non-existent
     *       objects (for security reasons, to prevent object enumeration)
     * </ul>

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Treat NoSuchUpload on abort as success (upload already gone) — log and continue instead of failing cleanup.
2. Grant s3:AbortMultipartUpload in the job's IAM policy alongside the put/multipart-list actions.
3. Retry once on transient 5xx before surfacing; abort is idempotent.
4. As a backstop for orphaned uploads, set a bucket lifecycle rule AbortIncompleteMultipartUpload (e.g. 7 days).

Example fix

// before — abort failure breaks cleanup
ops.abortMultiPartUpload(key, uploadId);

// after — tolerate already-gone uploads
try { ops.abortMultiPartUpload(key, uploadId); }
catch (IOException e) {
    if (!(e.getCause() instanceof NoSuchUploadException)) throw e;
    LOG.debug("Upload already completed/aborted: {} {}", key, uploadId);
}

catch (IOException e) { if (e.getCause() instanceof NoSuchUploadException) { LOG.debug("Upload already gone: {}", uploadId); /* treat as success */ } else if (((S3Exception) e.getCause()).statusCode() >= 500) { retryAbortOnce(); } else throw e; }

When it happens

Trigger: Aborting an uploadId that was already completed by a sibling attempt or already aborted (NoSuchUpload); aborting after the upload hit a lifecycle rule; IAM missing s3:AbortMultipartUpload on bucket/object; network/5xx during the abort call.

Common situations: Cleanup paths during failover racing a concurrent commit; error-handling code that aborts on any exception, including after a successful complete; restrictive IAM policies that grant PutObject but not AbortMultipartUpload.

Related errors

• Failed to start multipart upload for key: {}
• Cannot commit empty multipart upload for object: {}. This in
• Failed to upload part %d for key: %s, uploadId: %s
• Failed to put object for key: {}
• Failed to complete multipart upload for key: {}

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/d7666c504f38fafa. Report an issue: GitHub.