What causes "Multipart upload failed" in apache/hadoop?

Inspect getCause() of the IOException — it is the actual part-upload failure and dictates the fix

apache/hadoop · critical · IOException

Multipart upload failed

Error message

Multipart upload failed

What it means

At close(), BosOutputStream joins all UploadPartThreads, collecting any failure into uploadException; if one exists it calls abortMultipartUpload() — deleting every uploaded part server-side — and throws IOException('Multipart upload failed') with the part exception as cause. The object is never created; all written data is discarded.

Source

Thrown at hadoop-cloud-storage-project/hadoop-bos/src/main/java/org/apache/hadoop/fs/bos/BosOutputStream.java:404

        int index = 0;
        for (Future future : this.futureList) {
          future.get();
          index += 1;
          LOG.debug(
              "future.get() index: {} is done",
              index);
        }
      } catch (Exception e) {
        uploadException = e;
        LOG.warn(
            "catch exception when waiting"
                + " UploadPartThread done: ",
            e);
      }

      if (uploadException != null) {
        abortMultipartUpload();
        throw new IOException(
            "Multipart upload failed", uploadException);
      }

      LOG.debug(
          "success to wait upload part threads done");

      LOG.debug(
          "Size of eTags is {}. blkIndex is {}",
          this.eTags.size(), this.blkIndex);

      if (this.eTags.size() != this.blkIndex - 1) {
        abortMultipartUpload();
        throw new IOException(
            "Multipart upload incomplete: expected "
                + (this.blkIndex - 1) + " parts but got "
                + this.eTags.size());
      }

View on GitHub (pinned to 2add963021)

Solutions

Inspect getCause() of the IOException — it is the actual part-upload failure and dictates the fix
Re-run the write from the beginning: abort already removed partial parts and no object exists; the stream is not resumable
Make the producing job/checkpoint idempotent so the retry is safe at file granularity
For very large files, use credentials/STS TTLs longer than the worst-case write duration

Defensive patterns

Strategy: retry

Type guard

static boolean isMultipartUploadFailed(IOException e) {
  return "Multipart upload failed".equals(e.getMessage());
}

Try / catch

catch (IOException e) {
  if (isMultipartUploadFailed(e) && e.getCause() != null) {
    LOG.error("root cause", e.getCause());
    // abort already ran server-side; safe to recreate the object from scratch
    rewriteFromSource(path);
  } else { throw e; }
}

Prevention

Design commit-time retries: the abort leaves no partial object, so whole-file retry is safe
Use STS/credential TTL larger than worst-case upload duration
Alert on close()-phase failures — they discard the entire file

When it happens

Trigger: One or more part-upload threads fail during the final flush/close: network failure on the last parts, 403 from expired credentials, bucket deleted or ACL changed mid-write, sustained throttling.

Common situations: Job fails at commit stage after a long upload because the token expired before close; BOS maintenance window overlapping the write; bucket permissions changed while the job ran.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/839f21d08d47bdfb. Report an issue: GitHub.