apache/flink · error · IOException

Cannot commit empty multipart upload for object: {}. This in

Error message

Cannot commit empty multipart upload for object: {}. This indicates a programming error - at least one part must be uploaded before committing.

What it means

NativeS3Committer.commit() calls the S3 CompleteMultipartUpload API, which S3 rejects (and the AWS SDK models poorly) when the part list is empty. The code treats recoverable.parts().isEmpty() as a contract violation and throws IOException with 'programming error' wording: at least one part (even a zero-byte part for an empty file) must have been uploaded and recorded in the recoverable state before commit. Its javadoc explicitly says this signals a bug in the calling code or corruption of the persisted recoverable state.

Source

Thrown at flink-filesystems/flink-s3-fs-native/src/main/java/org/apache/flink/fs/s3native/writer/NativeS3Committer.java:66

            NativeS3ObjectOperations s3AccessHelper, NativeS3Recoverable recoverable) {
        this.s3AccessHelper = s3AccessHelper;
        this.recoverable = recoverable;
    }

    /**
     * Commits the multipart upload to finalize the S3 object.
     *
     * <p><b>Empty Parts Check:</b> Attempting to commit with no parts is considered a programming
     * error and will throw an IOException. This should not happen in normal operation as at least
     * one part must be uploaded before committing. If this exception is thrown, it indicates a bug
     * in the calling code or corruption of the recoverable state.
     *
     * @throws IOException if the commit fails or if attempting to commit with no parts
     */
    @Override
    public void commit() throws IOException {
        if (recoverable.parts().isEmpty()) {
            throw new IOException(
                    "Cannot commit empty multipart upload for object: "
                            + recoverable.getObjectName()
                            + ". This indicates a programming error - at least one part "
                            + "must be uploaded before committing.");
        }

        s3AccessHelper.commitMultiPartUpload(
                recoverable.getObjectName(),
                recoverable.uploadId(),
                recoverable.parts().stream()
                        .map(
                                part ->
                                        new NativeS3ObjectOperations.UploadPartResult(
                                                part.getPartNumber(), part.getETag()))
                        .collect(Collectors.toList()),
                recoverable.numBytesInParts());
    }

View on GitHub (pinned to 2f3c205e92)

Solutions

Ensure persist()/snapshot is only taken after at least one part exists: for an empty object, upload a single zero-byte part (the writer normally does this in flushForPersist) before persisting recoverable state.
If writing empty files is legitimate, verify the writer emitted the empty-part marker; if it did not, upgrade/patch the writer rather than catching this exception — it indicates no object will be committed.
Audit custom code that builds NativeS3Recoverable / calls commit() directly: pass the parts returned by uploadPart calls recorded in recoverable state.
If the recoverable state came from persistent storage and deserialized empty, treat the state as corrupt: discard the committable and re-run the write, and investigate why the parts list was lost.

Example fix

// before — persisting before any part exists
NativeS3Recoverable rec = writer.persistAfterRecovery(); // or captured pre-flush
new NativeS3Committer(rec, helper).commit(); // throws: empty parts

// after — flush so at least one part (possibly zero-byte) is uploaded, then persist
out.flush(); // uploads/pads the current part
NativeS3Recoverable rec = ((NativeS3RecoverableDataOutputStream) out).persist();
new NativeS3Committer(rec, helper).commit();

Defensive patterns

Strategy: validation

Validate before calling

// Never hand a possibly-empty recoverable to the committer
if (recoverable.parts().isEmpty()) {
    throw new IllegalStateException(
        "Refusing to commit with zero parts for " + recoverable.getObjectName()
        + " — flush the stream so at least one part exists before persisting");
}

Type guard

boolean isCommittable(NativeS3Recoverable r) { return r != null && r.parts() != null && !r.parts().isEmpty() && r.uploadId() != null; }

Try / catch

catch (IOException e) { if (e.getMessage()!=null && e.getMessage().contains("empty multipart upload")) { // programming error: do NOT retry; fix persist ordering / discard committable and re-run write } else throw e; }

Prevention

Only persist recoverable state after the writer flushed at least one part (empty files produce a zero-byte part).
Never construct NativeS3Recoverable by hand in user code or tests; use the RecoverableWriter's persist().
Treat this error as a bug report about the calling code, not an operational failure.

When it happens

Trigger: persist() captured after open() but before any data flush/part upload, then that recoverable state handed to commit(); a truncated/corrupted recoverable state file where the parts list deserialized as empty; a custom RecoverableWriter implementation or test harness that constructs NativeS3Recoverable with an empty parts list; edge case where a part-upload succeeded but the state persisted before recording it.

Common situations: Writing an empty file with a buggy flush/persist ordering; checkpointing a stream between open and first part; recovering a committable from an older savepoint whose parts field was not serialized; unit tests constructing recoverables by hand.

Related errors

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/0e9738c155789aff. Report an issue: GitHub.