apache/hadoop · error · IOException

Thread aborted

Error message

Thread aborted

What it means

ThrottledInputStream.throttle() sleeps in a loop whenever the observed transfer rate exceeds the cap (set by -bandwidth / distcp.map.bandwidth.mb). If the reading thread is interrupted during that sleep, the InterruptedException is converted to IOException("Thread aborted", e). It signals the MapReduce task was cancelled or killed mid-read, not a data problem.

Source

Thrown at hadoop-tools/hadoop-distcp/src/main/java/org/apache/hadoop/tools/util/ThrottledInputStream.java:104

  public int read(byte[] b, int off, int len) throws IOException {
    if (len == 0) {
      return 0;
    }
    throttle();
    int readLen = rawStream.read(b, off, len);
    if (readLen != -1) {
      bytesRead += readLen;
    }
    return readLen;
  }

  private void throttle() throws IOException {
    while (getBytesPerSec() > maxBytesPerSec) {
      try {
        Thread.sleep(SLEEP_DURATION_MS);
        totalSleepTime += SLEEP_DURATION_MS;
      } catch (InterruptedException e) {
        throw new IOException("Thread aborted", e);
      }
    }
  }

  /**
   * Getter for the number of bytes read from this stream, since creation.
   * @return The number of bytes.
   */
  public long getTotalBytesRead() {
    return bytesRead;
  }

  /**
   * Getter for the read-rate from this stream, since creation.
   * Calculated as bytesRead/elapsedTimeSinceStart.
   * @return Read rate, in bytes/sec.
   */
  public long getBytesPerSec() {

View on GitHub (pinned to 2add963021)

Solutions

1. Treat it as a cancellation signal: check JobHistory/ApplicationMaster logs for why the attempt was killed
2. No data repair needed: the framework retries live attempts; re-run the job if it was externally killed
3. If embedding ThrottledInputStream, catch the IOException and restore the interrupt flag via Thread.currentThread().interrupt()
4. Do not manually interrupt threads that own filesystem streams

Example fix

// before: interrupt status silently swallowed
try {
  while ((n = throttledIn.read(buf)) != -1) { /* write */ }
} catch (IOException e) {
  log.error("copy failed", e);
}

// after: honour cancellation
try {
  while ((n = throttledIn.read(buf)) != -1) { /* write */ }
} catch (IOException e) {
  if (e.getCause() instanceof InterruptedException) {
    Thread.currentThread().interrupt();   // preserve cancellation
  } else {
    throw e;
  }
}

catch (IOException e) {
  if (e.getCause() instanceof InterruptedException) {
    Thread.currentThread().interrupt();  // propagate cancellation correctly
  } else {
    throw e;
  }
}

When it happens

Trigger: The MR task attempt is killed by the framework (preemption, speculative-execution loser, job kill/abort) while the throttled stream was sleeping in throttle(); externally interrupting a thread that reads through ThrottledInputStream in library usage.

Common situations: Operator-cancelled jobs; YARN preemption; speculative tasks being stopped late; test harnesses interrupting worker threads.

Related errors

• status code 429 !!!" + e.getCause()
• key + ": No such file or directory."
• Exceeded limit of max asynchronous calls: %d, please configu
• Call interrupted
• Bandwidth specified is invalid: {value}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/c0b4dc6da35e12a6. Report an issue: GitHub.