{"record":{"id":"49f6b1859a3e3568","repo":"apache/hadoop","slug":"blockchecksumtype-composite-crc-doesn-t-support-mi","errorCode":null,"errorMessage":"BlockChecksumType COMPOSITE_CRC doesn't support MIXED underlying types; previous block was %s, next block is %s","messagePattern":"BlockChecksumType COMPOSITE_CRC doesn't support MIXED underlying types; previous block was (.+?), next block is (.+?)","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockChecksumHelper.java","lineNumber":752,"sourceCode":"        setBytesPerCRC(bpc);\n      } else if (bpc != getBytesPerCRC()) {\n        throw new IOException(\"Byte-per-checksum not matched: bpc=\" + bpc\n            + \" but bytesPerCRC=\" + getBytesPerCRC());\n      }\n\n      //read crc-per-block\n      if (blockIdx == 0) {\n        setCrcPerBlock(cpb);\n      }\n\n      if (blockIdx == 0) { // first block\n        setCrcType(ct);\n      } else if (getCrcType() != DataChecksum.Type.MIXED &&\n          getCrcType() != ct) {\n        BlockChecksumType groupChecksumType =\n            getBlockChecksumOptions().getBlockChecksumType();\n        if (groupChecksumType == BlockChecksumType.COMPOSITE_CRC) {\n          throw new IOException(String.format(\n              \"BlockChecksumType COMPOSITE_CRC doesn't support MIXED \"\n              + \"underlying types; previous block was %s, next block is %s\",\n              getCrcType(), ct));\n        } else {\n          setCrcType(DataChecksum.Type.MIXED);\n        }\n      }\n\n      if (blockIdx == 0) {\n        LOG.debug(\"set bytesPerCRC={}, crcPerBlock={}\", getBytesPerCRC(),\n            getCrcPerBlock());\n      }\n    }\n  }\n}\n","sourceCodeStart":734,"sourceCodeEnd":768,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/datanode/BlockChecksumHelper.java#L734-L768","documentation":"COMPOSITE_CRC requires the underlying CRC algorithm (DataChecksum.Type, e.g. CRC32 vs CRC32C) to be identical across all internal blocks of the group; the Nth block reported a different type. Unlike MD5CRC, composite folding cannot represent a MIXED group, so it throws instead of silently producing a wrong digest.","triggerScenarios":"The checksum type used at write time (per-file create option / dfs.checksum.type) differed between internal blocks of one striped file — e.g. type changed between original write and a later append, or blocks written by DNs with different configs.","commonSituations":"Changing dfs.checksum.type on an existing cluster; appending to old files after a config change; EC files written across a config migration.","solutions":["Use MD5CRC for checksumming the affected file (MIXED is representable there)","Keep the checksum type constant cluster-wide and never change it for files that will be appended","Rewrite affected files (distcp) so all internal blocks share one type"],"exampleFix":null,"handlingStrategy":"fallback","validationCode":"// Prefer MD5CRC for files that could predate a checksum-type change:\nBlockChecksumOptions opts = fileMayPredateConfigChange\n    ? MD5CRC_OPTS : COMPOSITE_CRC_OPTS;","typeGuard":null,"tryCatchPattern":"try {\n  cs = fs.getFileChecksum(path, COMPOSITE_CRC_OPTS);\n} catch (IOException e) {\n  if (e.getMessage().contains(\"doesn't support MIXED\")) {\n    cs = fs.getFileChecksum(path, MD5CRC_OPTS);\n  } else { throw e; }\n}","preventionTips":["Never change the checksum type on a cluster whose files may later be appended","Run fileChecksum with MD5CRC as the compatibility baseline in verification tools"],"tags":["hdfs","erasure-coding","composite-crc","configuration-mismatch"],"backgroundTag":"checksum-config-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}