apache/hadoop · error · IOException

Corrupted block: ${this}

Error message

Corrupted block: ${this}

What it means

Error "Corrupted block: ${this}" thrown in apache/hadoop.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/LocalReplicaInPipeline.java:335

        checksum = header.getChecksum();

        if (checksum.getBytesPerChecksum() !=
            requestedChecksum.getBytesPerChecksum()) {
          throw new IOException("Client requested checksum " +
              requestedChecksum + " when appending to an existing block " +
              "with different chunk size: " + checksum);
        }

        int bytesPerChunk = checksum.getBytesPerChecksum();
        int checksumSize = checksum.getChecksumSize();

        blockDiskSize = bytesOnDisk;
        crcDiskSize = BlockMetadataHeader.getHeaderSize() +
          (blockDiskSize+bytesPerChunk-1)/bytesPerChunk*checksumSize;
        if (blockDiskSize > 0 &&
            (blockDiskSize > blockFile.length() ||
               crcDiskSize>metaFile.length())) {
          throw new IOException("Corrupted block: " + this);
        }
        checkedMeta = true;
      } finally {
        if (!checkedMeta) {
          // clean up in case of exceptions.
          IOUtils.closeStream(metaRAF);
        }
      }
    } else {
      // for create, we can use the requested checksum
      checksum = requestedChecksum;
    }

    final FileIoProvider fileIoProvider = getFileIoProvider();
    FileOutputStream blockOut = null;
    FileOutputStream crcOut = null;
    try {
      blockOut = fileIoProvider.getFileOutputStream(

View on GitHub (pinned to 2add963021)

Solutions

1. Report the block as corrupt (the DataNode normally does this automatically) so the NameNode re-replicates from a good replica.
2. Check disk/memory health on this DataNode to find the corruption source.

When it happens

Trigger: A CRC check on the replica's data or metadata fails while the replica is in the pipeline, marking it corrupt.

Common situations: The replica being written failed validation and was marked corrupt. Check disk health and the client pipeline for the corruption source.

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/ab1435bb6292a427. Report an issue: GitHub.