apache/hadoop · error · InterruptedIOException

Interrupted while copying objects (copy)

Error message

Interrupted while copying objects (copy)

What it means

During an object-bucket rename, each source object is copied by a task submitted to a thread pool; waitAllCopyFinished blocks on every Future. If the waiting thread is interrupted, the code logs a warning and throws InterruptedIOException('Interrupted while copying objects (copy)'). The rename stops mid-way, so some destination objects may already exist while the source is still intact.

Source

Thrown at hadoop-cloud-storage-project/hadoop-huaweicloud/src/main/java/org/apache/hadoop/fs/obs/OBSObjectBucketUtils.java:376

              + srcKey
              + ", dst="
              + dstKey
              + ", delay="
              + delay
              + "}");
    }
  }

  private static void waitAllCopyFinished(
      final List<Future<CopyObjectResult>> copyFutures)
      throws IOException {
    try {
      for (Future<CopyObjectResult> copyFuture : copyFutures) {
        copyFuture.get();
      }
    } catch (InterruptedException e) {
      LOG.warn("Interrupted while copying objects (copy)");
      throw new InterruptedIOException(
          "Interrupted while copying objects (copy)");
    } catch (ExecutionException e) {
      for (Future<CopyObjectResult> future : copyFutures) {
        future.cancel(true);
      }

      throw OBSCommonUtils.extractException(
          "waitAllCopyFinished", copyFutures.toString(), e);
    }
  }

  /**
   * Request object metadata; increments counters in the process.
   *
   * @param owner OBS File System instance
   * @param key   key
   * @return the metadata
   */

View on GitHub (pinned to 2add963021)

Solutions

1. Do not interrupt the worker thread: cancel jobs only at safe points, or let the rename finish
2. Catch InterruptedIOException, restore the interrupt flag with Thread.currentThread().interrupt(), clean the partial destination, and re-run the rename
3. Split very large directory moves into smaller subtrees so a timeout is less likely to fire mid-copy

Example fix

// before
Future<?> f = pool.submit(() -> fs.rename(src, dst));
f.get(1, TimeUnit.SECONDS); // timeout -> cancel(true) -> InterruptedIOException

// after
Future<?> f = pool.submit(() -> fs.rename(src, dst));
f.get(); // let the copy finish; interrupt only on shutdown

// recovery path
try {
  fs.rename(src, dst);
} catch (InterruptedIOException e) {
  Thread.currentThread().interrupt();
  fs.delete(dst, true);
  fs.rename(src, dst); // re-run after cleanup
}

try {
  fs.rename(src, dst);
} catch (InterruptedIOException e) {
  Thread.currentThread().interrupt(); // restore the flag
  fs.delete(dst, true);              // remove partial copies
  fs.rename(src, dst);               // re-run: source is still intact
}

When it happens

Trigger: The thread running FileSystem.rename is interrupted: YARN or Spark cancels the task (speculative execution, preemption), executor shutdownNow(), a wrapping Future.cancel(true), or a caller timeout that aborts the operation.

Common situations: Speculative tasks cancelled mid-commit; user cancels a long job moving a large directory; frameworks with hard per-task timeouts interrupting a big copy phase; shutdown hooks racing in-flight renames.

Related errors

• {} is root directory
• new destination is an existed directory
• new destination is an existed file
• destination is an existed file
• destination parent [{}] is not a directory

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/00e777f690aa4407. Report an issue: GitHub.