apache/hadoop · error · InterruptedIOException

Interrupted while waiting for reading task

Error message

Interrupted while waiting for reading task

What it means

Hedged-read path: while polling the first read future (dfs.client.hedged.read.threshold.millis elapsed, second read spawned), an InterruptedException from future.get() is converted to InterruptedIOException, aborting the hedged read loop. Pure thread-cancellation semantics; the data and cluster state are not implicated.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/DFSInputStream.java:1363

        futures.add(firstRequest);
        Future<ByteBuffer> future = null;
        try {
          future = hedgedService.poll(
              conf.getHedgedReadThresholdMillis(), TimeUnit.MILLISECONDS);
          if (future != null) {
            ByteBuffer result = future.get();
            result.flip();
            buf.put(result);
            return;
          }
          DFSClient.LOG.debug("Waited {}ms to read from {}; spawning hedged "
              + "read", conf.getHedgedReadThresholdMillis(), chosenNode.info);
          dfsClient.getHedgedReadMetrics().incHedgedReadOps();
          // continue; no need to refresh block locations
        } catch (ExecutionException e) {
          futures.remove(future);
        } catch (InterruptedException e) {
          throw new InterruptedIOException(
              "Interrupted while waiting for reading task");
        }
        // Ignore this node on next go around.
        // If poll timeout and the request still ongoing, don't consider it
        // again. If read data failed, don't consider it either.
        ignored.add(chosenNode.info);
      } else {
        // We are starting up a 'hedged' read. We have a read already
        // ongoing. Call getBestNodeDNAddrPair instead of chooseDataNode.
        // If no nodes to do hedged reads against, pass.
        boolean refetch = false;
        try {
          chosenNode = chooseDataNode(block, ignored, false);
          if (chosenNode != null) {
            // Latest block, if refreshed internally
            block = chosenNode.block;
            bb = ByteBuffer.allocate(len);
            Callable<ByteBuffer> getFromDataNodeCallable =

View on GitHub (pinned to 2add963021)

Solutions

1. Propagate as cancellation; do not retry in the interrupted thread
2. Cancel readers by closing the stream instead of interrupting threads
3. Only enable hedged reads on pools you never interrupt mid-read

static boolean isCancelled(IOException e) {
  return e instanceof InterruptedIOException;
}

try {
  n = in.read(buf);
} catch (InterruptedIOException e) {
  // cancelled while hedged reads were in flight: unwind cleanly
  throw new java.util.concurrent.CancellationException("hedged read interrupted", e);
}

When it happens

Trigger: Thread interrupt while hedged reads are enabled (dfs.client.hedged.read.threadpool.size > 0) and the reader waits on a CompletableFuture result.

Common situations: Task/job cancellation landing while hedged reads are in flight on slow clusters; shutdownNow() on reader thread pools.

Related errors

• Interrupted while getting the last block length.
• Interrupted while getting the length.
• Interrupted while copying objects (copy)
• Another {name} is running.
• Commit or complete block {commitBlock}, whereas it is under

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/4b6fc2cf7c0921a7. Report an issue: GitHub.