What causes "Interrupted multi-part upload with id '%s' to %s" in apache/hadoop?

Avoid interrupting writer threads during commit: use graceful cancellation (let the current write/close finish, or cancel between files).

apache/hadoop · error · IOException

Interrupted multi-part upload with id '%s' to %s

Error message

Interrupted multi-part upload with id '%s' to %s

What it means

OBSBlockOutputStream's block-upload wait does Futures.allAsList(partETagsFutures).get(); an InterruptedException there means some other thread called Thread.interrupt() on the writer while it was joining part uploads. The stream then cancels all part futures (future.cancel(true)), calls this.abort() to kill the multipart upload, and throws IOException('Interrupted multi-part upload with id '<id>' to <key>'). No data is committed: the MPU is aborted and the file must be rewritten from scratch.

Source

Thrown at hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSBlockOutputStream.java:751

     * Block awaiting all outstanding uploads to complete.
     *
     * @return list of results
     * @throws IOException IO Problems
     */
    private List<PartEtag> waitForAllPartUploads() throws IOException {
      LOG.debug("Waiting for {} uploads to complete",
          partETagsFutures.size());
      try {
        return Futures.allAsList(partETagsFutures).get();
      } catch (InterruptedException ie) {
        LOG.warn("Interrupted partUpload", ie);
        LOG.debug("Cancelling futures");
        for (ListenableFuture<PartEtag> future : partETagsFutures) {
          future.cancel(true);
        }
        // abort multipartupload
        this.abort();
        throw new IOException(
            "Interrupted multi-part upload with id '" + uploadId
                + "' to " + key);
      } catch (ExecutionException ee) {
        // there is no way of recovering so abort
        // cancel all partUploads
        LOG.debug("While waiting for upload completion", ee);
        LOG.debug("Cancelling futures");
        for (ListenableFuture<PartEtag> future : partETagsFutures) {
          future.cancel(true);
        }
        // abort multipartupload
        this.abort();
        throw OBSCommonUtils.extractException(
            "Multi-part upload with id '" + uploadId + "' to " + key,
            key, ee);
      }
    }

View on GitHub (pinned to 2add963021)

Solutions

Avoid interrupting writer threads during commit: use graceful cancellation (let the current write/close finish, or cancel between files).
If interruption is legitimate (task preemption), simply accept the abort and re-run the task — the output file never existed, so the job can safely retry it idempotently.
Replace shutdownNow() with shutdown() + awaitTermination so in-flight uploads drain.
Make writes idempotent (write to temp path then atomic rename) so a retried task after cancellation does not duplicate data.

Example fix

// before
ExecutorService pool = ...;
// task uses pool.submit(writerTask) ...
pool.shutdownNow(); // interrupts writer waiting on partETagsFutures -> MPU aborted

// after
pool.shutdown();
if (!pool.awaitTermination(60, TimeUnit.SECONDS)) {
  log.warn("writers did not finish; data may need re-run");
  pool.shutdownNow();
}

Defensive patterns

Strategy: retry

Validate before calling

// nothing to validate client-side; instead make cancellation graceful
boolean drained = pool.awaitTerminationGracefully(); // shutdown() + awaitTermination before any cancel

Try / catch

try {
  out.close(); // commit waits on partETagsFutures
} catch (IOException e) {
  if (String.valueOf(e.getMessage()).startsWith("Interrupted multi-part upload")) {
    LOG.warn("commit interrupted; task will be retried — output was aborted, safe to re-run", e);
    throw new RetryableTaskException(e); // scheduler re-executes the whole write
  }
  throw e;
}

Prevention

Use shutdown()+awaitTermination, never shutdownNow() on pools that write.
Make writes idempotent: temp path + atomic rename on success.
Treat task preemption during commit as retryable, not as data loss — the MPU abort guarantees no partial object.
Avoid Future.cancel(true) on tasks holding output streams.

When it happens

Trigger: Job/task cancellation (YARN preemption, Spark job cancellation) interrupting the committing thread; ExecutorService.shutdownNow() while a writer awaits part uploads; user code calling Thread.stop-style interrupts or Future.cancel(true) on the writing task; CLI tools killed with certain signals that surface as interrupts.

Common situations: Spark speculation killing speculative tasks mid-commit; Flink checkpoint timeouts cancelling sink writers; test harnesses that shutdownNow() executors in finally blocks; monitoring scripts killing long uploads.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/3b19c893e7fa14c8. Report an issue: GitHub.