{"record":{"id":"cab9716f417b79e4","repo":"apache/hadoop","slug":"unexpected-blockchecksumtype-s-expecting-compo","errorCode":null,"errorMessage":"Unexpected blockChecksumType '%s', expecting COMPOSITE_CRC","messagePattern":"Unexpected blockChecksumType '(.+?)', expecting COMPOSITE_CRC","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockChecksumHelper.java","lineNumber":667,"sourceCode":"        }\n\n        setOrVerifyChecksumProperties(blockIdx, checksumData.getBytesPerCrc(),\n            checksumData.getCrcPerBlock(), ct);\n\n        switch (groupChecksumType) {\n        case MD5CRC:\n          //read md5\n          final MD5Hash md5 =\n              new MD5Hash(checksumData.getBlockChecksum().toByteArray());\n          md5.write(blockChecksumBuf);\n          LOG.debug(\"got reply from datanode:{}, md5={}\",\n              targetDatanode, md5);\n          break;\n        case COMPOSITE_CRC:\n          BlockChecksumType returnedType = PBHelperClient.convert(\n              checksumData.getBlockChecksumOptions().getBlockChecksumType());\n          if (returnedType != BlockChecksumType.COMPOSITE_CRC) {\n            throw new IOException(String.format(\n                \"Unexpected blockChecksumType '%s', expecting COMPOSITE_CRC\",\n                returnedType));\n          }\n          byte[] checksumBytes =\n              checksumData.getBlockChecksum().toByteArray();\n          blockChecksumBuf.write(checksumBytes, 0, checksumBytes.length);\n          if (LOG.isDebugEnabled()) {\n            LOG.debug(\"got reply from datanode:{} for blockIdx:{}, checksum:{}\",\n                targetDatanode, blockIdx,\n                CrcUtil.toMultiCrcString(checksumBytes));\n          }\n          break;\n        default:\n          throw new IOException(\n              \"Unknown BlockChecksumType: \" + groupChecksumType);\n        }\n      }\n    }","sourceCodeStart":649,"sourceCodeEnd":685,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockChecksumHelper.java#L649-L685","documentation":"During a COMPOSITE_CRC block-group checksum the DataNode asked a peer DN for a per-block composite CRC, but the reply's echoed BlockChecksumOptions reported a different type. The helper validates the echoed type because an older peer silently answers MD5CRC instead of honoring the requested COMPOSITE_CRC.","triggerScenarios":"An EC file's block group spans a DataNode running a release without COMPOSITE_CRC support, so it ignores the requested type and returns MD5CRC; also possible with a corrupted or hand-crafted BlockOpResponseProto.","commonSituations":"Rolling upgrade mid-way across DNs hosting one striped file; mixed-version cluster; distcp/verification tooling requesting composite CRCs too early.","solutions":["Finish the rolling upgrade so every DataNode supports COMPOSITE_CRC before clients request it","Until then, run fileChecksum with MD5CRC (set dfs.client.files.checksum-type or the client option) — it works on mixed versions","If versions are already aligned, inspect the peer DN for protocol corruption or a stale jar"],"exampleFix":"// before\nBlockChecksumOptions opts =\n    new BlockChecksumOptions(BlockChecksumType.COMPOSITE_CRC, cellSize);\n\n// after (until all DNs are upgraded)\nBlockChecksumOptions opts =\n    new BlockChecksumOptions(BlockChecksumType.MD5CRC, cellSize);","handlingStrategy":"try-catch","validationCode":"// Before requesting COMPOSITE_CRC on an EC file, confirm every replica-hosting\n// DN runs a release that supports it (e.g. via DatanodeInfo version):\nfor (DatanodeInfo dn : locatedBlock.getLocations()) {\n  if (!supportsCompositeCrc(dn.getSoftwareVersion())) {\n    useComposite = false; break;\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  digest = getFileChecksum(ecPath, COMPOSITE_CRC);\n} catch (IOException e) {\n  if (e.getMessage().contains(\"expecting COMPOSITE_CRC\")) {\n    digest = getFileChecksum(ecPath, MD5CRC); // mixed-version safe\n  } else { throw e; }\n}","preventionTips":["Finish rolling upgrades before enabling composite CRC clients","Verify DN versions (jmx / DatanodeInfo) match across the EC block group when checksum verification matters"],"tags":["hdfs","datanode","composite-crc","version-skew","data-transfer-protocol"],"backgroundTag":"client-server-version-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}