{"record":{"id":"71f2e734e3197510","repo":"apache/hadoop","slug":"could-not-create-datachecksum-of-type-d-with-byte","errorCode":null,"errorMessage":"Could not create DataChecksum of type %d with bytesPerChecksum %d","messagePattern":"Could not create DataChecksum of type (.+?) with bytesPerChecksum (.+?)","errorType":"exception","errorClass":"InvalidChecksumSizeException","httpStatus":null,"severity":"critical","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java","lineNumber":174,"sourceCode":"    return csum;\n  }\n  \n  /**\n   * This constructs a DataChecksum by reading HEADER_LEN bytes from input\n   * stream <i>in</i>.\n   *\n   * @param in data input stream.\n   * @throws IOException raised on errors performing I/O.\n   * @return DataChecksum by reading HEADER_LEN\n   *         bytes from input stream.\n   */\n  public static DataChecksum newDataChecksum( DataInputStream in )\n                                 throws IOException {\n    int type = in.readByte();\n    int bpc = in.readInt();\n    DataChecksum summer = newDataChecksum(mapByteToChecksumType(type), bpc);\n    if ( summer == null ) {\n      throw new InvalidChecksumSizeException(\"Could not create DataChecksum \"\n          + \"of type \" + type + \" with bytesPerChecksum \" + bpc);\n    }\n    return summer;\n  }\n\n  private static Type mapByteToChecksumType(int type)\n      throws InvalidChecksumSizeException{\n    try {\n      return Type.valueOf(type);\n    } catch (IllegalArgumentException e) {\n      throw new InvalidChecksumSizeException(\"The value \"+type+\" does not map\"+\n        \" to a valid checksum Type\");\n    }\n  }\n  \n  /**\n   * Writes the checksum header to the output stream <i>out</i>.\n   *","sourceCodeStart":156,"sourceCodeEnd":192,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/DataChecksum.java#L156-L192","documentation":"DataChecksum.newDataChecksum(DataInputStream) reads the 1-byte checksum type and 4-byte bytesPerChecksum directly from the wire (block data transfer and checksum-reading paths). If no checksummer exists for that (type, bytesPerChecksum) pair, it throws InvalidChecksumSizeException echoing both integers — the stream did not contain a valid checksum header where one was expected.","triggerScenarios":"The stream is positioned at the wrong offset (misaligned read of block checksum data); a peer writing a different data layout; corrupted checksum header bytes in block metadata being read back.","commonSituations":"Reading block metadata written by an incompatible Hadoop version; stream positioning bugs after partial reads; storage corruption in the checksum header region.","solutions":["Verify the stream is positioned at a checksum header before calling (read the 5 header bytes deliberately)","Check block metadata version compatibility between writer and reader (BlockMetadataHeader layout)","Run `hdfs fsck` and re-replicate the corrupted block","On version skew, finish the rolling upgrade before reading affected data with older components"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n  DataChecksum csum = DataChecksum.newDataChecksum(in);\n} catch (InvalidChecksumSizeException e) {\n  // stream not at a valid checksum header: realign, re-open, or fail over to another replica\n  reopenStreamAtLastBoundary();\n}","preventionTips":["Read block metadata through BlockMetadataHeader helpers instead of raw stream reads","Complete rolling upgrades before old components read new-format checksums","Unit-test stream positioning whenever you touch block-reading code"],"tags":["hdfs","checksum","data-integrity","stream","datachecksum"],"backgroundTag":"data-corruption-detected","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}