apache/hadoop · error · IOException

Unknown BlockChecksumType: " + groupChecksumType

Error message

Unknown BlockChecksumType: " + groupChecksumType

What it means

While preparing per-internal-block child requests for an erasure-coded block-group checksum, the DataNode switches on the group checksum type to build child BlockChecksumOptions (MD5CRC passes the parent options through; COMPOSITE_CRC is rewritten with the EC cell size). Any other value hits the default and aborts before any peer request is sent.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockChecksumHelper.java:625

            getDatanode(), Op.BLOCK_CHECKSUM, block);

        // get block checksum
        // A BlockGroupCheckum of type COMPOSITE_CRC uses underlying
        // BlockChecksums also of type COMPOSITE_CRC but with
        // stripeLength == ecPolicy.getCellSize().
        BlockChecksumOptions childOptions;
        BlockChecksumType groupChecksumType =
            getBlockChecksumOptions().getBlockChecksumType();
        switch (groupChecksumType) {
        case MD5CRC:
          childOptions = getBlockChecksumOptions();
          break;
        case COMPOSITE_CRC:
          childOptions = new BlockChecksumOptions(
              BlockChecksumType.COMPOSITE_CRC, ecPolicy.getCellSize());
          break;
        default:
          throw new IOException(
              "Unknown BlockChecksumType: " + groupChecksumType);
        }
        createSender(pair).blockChecksum(block, blockToken, childOptions);

        final DataTransferProtos.BlockOpResponseProto reply =
            DataTransferProtos.BlockOpResponseProto.parseFrom(
                PBHelperClient.vintPrefixed(pair.in));

        String logInfo = "for block " + block
            + " from datanode " + targetDatanode;
        DataTransferProtoUtil.checkBlockOpStatus(reply, logInfo);

        DataTransferProtos.OpBlockChecksumResponseProto checksumData =
            reply.getChecksumResponse();

        // read crc-type
        final DataChecksum.Type ct;
        if (checksumData.hasCrcType()) {

View on GitHub (pinned to 2add963021)

Solutions

1. Use only MD5CRC or COMPOSITE_CRC for striped files until the whole cluster is on one version
2. Finish the rolling upgrade before any client requests composite CRCs on EC data
3. Check the client's checksum-type setting that feeds BlockChecksumOptions

BlockChecksumType t = opts.getBlockChecksumType();
if (t != BlockChecksumType.MD5CRC && t != BlockChecksumType.COMPOSITE_CRC) {
  opts = new BlockChecksumOptions(BlockChecksumType.MD5CRC,
      opts.getStripeLength());
}

static boolean isKnownBlockChecksumType(BlockChecksumType t) {
  return t == BlockChecksumType.MD5CRC
      || t == BlockChecksumType.COMPOSITE_CRC;
}

catch (IOException e) {
  if (String.valueOf(e.getMessage()).contains("Unknown BlockChecksumType")) {
    LOG.warn("DN lacks support for {}; retrying as MD5CRC", requestedType);
    return fetchAsMd5Crc();
  }
  throw e;
}

When it happens

Trigger: getBlockChecksum/fileChecksum on a striped (EC) block group whose requested BlockChecksumType is neither MD5CRC nor COMPOSITE_CRC — again typically an unknown enum ordinal from version skew or manual construction.

Common situations: Rolling upgrade with EC files in play, custom client builds, cross-branch testing.

Related errors

• Unrecognized BlockChecksumType: %s
• {} has no enough internal blocks(current: {}), unable to sta
• Fail to get block checksum for {}
• {} missing blocks, the stripe is: {}; locatedBlocks is: {}
• Unexpected EOS from the reader

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/df75fcc5378f4aef. Report an issue: GitHub.