apache/hadoop · error · IOException

Shutting down writer and responder due to a checksum error i

Error message

Shutting down writer and responder due to a checksum error in received data. The error response has been sent upstream.

What it means

Error "Shutting down writer and responder due to a checksum error in received data. The error response has been sent upstream." thrown in apache/hadoop.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockReceiver.java:1703

          inAddr,
          duration);
      if (duration > datanodeSlowLogThresholdMs) {
        datanode.metrics.incrSlowAckToUpstreamCount();
        LOG.warn("Slow PacketResponder send ack to upstream took " + duration
            + "ms (threshold=" + datanodeSlowLogThresholdMs + "ms), " + myString
            + ", replyAck=" + replyAck
            + ", downstream DNs=" + Arrays.toString(downstreamDNs)
            + ", blockId=" + replicaInfo.getBlockId()
            + ", seqno=" + seqno);
      } else if (LOG.isDebugEnabled()) {
        LOG.debug(myString + ", replyAck=" + replyAck);
      }

      // If a corruption was detected in the received data, terminate after
      // sending ERROR_CHECKSUM back.
      Status myStatus = PipelineAck.getStatusFromHeader(myHeader);
      if (myStatus == Status.ERROR_CHECKSUM) {
        throw new IOException("Shutting down writer and responder "
            + "due to a checksum error in received data. The error "
            + "response has been sent upstream.");
      }
    }
    
    /**
     * Remove a packet from the head of the ack queue
     *
     * This should be called only when the ack queue is not empty
     */
    private void removeAckHead() {
      synchronized (ackQueue) {
        ackQueue.remove();
        ackQueue.notifyAll();
      }
    }
  }

View on GitHub (pinned to 2add963021)

Solutions

1. Check local disk and RAM health on this DataNode, since it detected a checksum error in data it received.
2. Let the pipeline recovery continue; the block will be rewritten through a new pipeline and the bad replica reported to the NameNode.

When it happens

Trigger: The DataNode detects a checksum error while receiving block data, notifies upstream, and tears down the writer and responder for that block.

Common situations: A checksum error in received data forced the writer and responder to shut down after notifying upstream. Check local disk and memory health.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/d88886142071cff1. Report an issue: GitHub.