apache/hadoop · error · IOException
Join on responder thread {} timed out
Error message
Join on responder thread {} timed out What it means
Error "Join on responder thread {} timed out" thrown in apache/hadoop.
Source
Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java:1105
responder.interrupt();
}
IOUtils.closeStream(this);
cleanupBlock();
}
if (responder != null) {
try {
responder.interrupt();
// join() on the responder should timeout a bit earlier than the
// configured deadline. Otherwise, the join() on this thread will
// likely timeout as well.
long joinTimeout = datanode.getDnConf().getXceiverStopTimeout();
joinTimeout = joinTimeout > 1 ? joinTimeout*8/10 : joinTimeout;
responder.join(joinTimeout);
if (responder.isAlive()) {
String msg = "Join on responder thread " + responder
+ " timed out";
LOG.warn(msg + "\n" + StringUtils.getStackTrace(responder));
throw new IOException(msg);
}
} catch (InterruptedException e) {
responder.interrupt();
// do not throw if shutting down for restart.
if (!datanode.isRestarting()) {
throw new InterruptedIOException("Interrupted receiveBlock");
}
}
responder = null;
}
}
}
/**
* If we have downstream DNs and peerMetrics are enabled, then initialize
* some state for monitoring the performance of downstream DNs.
*
* @param downstreams downstream DNs, or null if there are none.View on GitHub (pinned to 2add963021)
Solutions
- Increase dfs.datanode.handler.count or investigate why the PacketResponder thread is stuck (slow disk, blocked downstream mirror).
- Check downstream DataNode logs and disk health; a hung responder usually indicates the write pipeline is stalled.
When it happens
Trigger: The receiver thread waits to join the PacketResponder thread after the write finished or failed, but the responder does not exit within the timeout.
Common situations: The PacketResponder thread did not finish within the join timeout during block write shutdown. Look for blocked downstream DataNodes or slow disks delaying acks.
Understand the failure class
- Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.
AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22).
Data as JSON: /api/errors/63d4b9569a69e160.
Report an issue: GitHub.