{"record":{"id":"609e2ed7b8ae1814","repo":"apache/hadoop","slug":"blocksize-should-be-a-multiple-of-checksumsize","errorCode":null,"errorMessage":"blockSize should be a multiple of checksumsize","messagePattern":"blockSize should be a multiple of checksumsize","errorType":"validation","errorClass":"HadoopIllegalArgumentException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java","lineNumber":634,"sourceCode":"    // checksumOpt. Any missing value will be filled in using the default.\n    ChecksumOpt defaultOpt = new ChecksumOpt(\n        ssDef.getChecksumType(),\n        ssDef.getBytesPerChecksum());\n    checksumOpt = ChecksumOpt.processChecksumOpt(defaultOpt,\n        checksumOpt, bytesPerChecksum);\n\n    if (bufferSize == -1) {\n      bufferSize = ssDef.getFileBufferSize();\n    }\n    if (replication == -1) {\n      replication = ssDef.getReplication();\n    }\n    if (createParent == null) {\n      createParent = false;\n    }\n\n    if (blockSize % bytesPerChecksum != 0) {\n      throw new HadoopIllegalArgumentException(\n             \"blockSize should be a multiple of checksumsize\");\n    }\n\n    return this.createInternal(f, createFlag, permission, bufferSize,\n      replication, blockSize, progress, checksumOpt, createParent);\n  }\n\n  /**\n   * The specification of this method matches that of\n   * {@link #create(Path, EnumSet, Options.CreateOpts...)} except that the opts\n   * have been declared explicitly.\n   *\n   * @param f the path.\n   * @param flag create flag.\n   * @param absolutePermission absolute permission.\n   * @param bufferSize buffer size.\n   * @param replication replications.\n   * @param blockSize block size.","sourceCodeStart":616,"sourceCodeEnd":652,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java#L616-L652","documentation":"Final argument validation in AbstractFileSystem.create: the resolved blockSize (user option or server default) must be divisible by the resolved bytesPerChecksum (user option or the 512 default), else HadoopIllegalArgumentException. Unlike the server-defaults internal check, this validates the merged values the caller actually influenced.","triggerScenarios":"CreateOpts.blockSize(1_000_000) with the default 512-byte checksum; combining CreateOpts.blockSize(...) with CreateOpts.bytesPerChecksum(1024) where the former is not a multiple of the latter; computing block size from payload/quota sizes and passing it raw.","commonSituations":"Apps deriving blockSize from record counts or user settings; switching bytesPerChecksum for performance without adjusting block size; small test files created with tiny odd block sizes.","solutions":["Round the block size to a multiple of bytesPerChecksum (typically 512) before passing it — e.g. round up: bs = ((bs + bpc - 1) / bpc) * bpc","Omit CreateOpts.blockSize and use the server default, which is already consistent","When changing io.bytes.per.checksum cluster-side, audit code that passes explicit block sizes"],"exampleFix":"// before\nfc.create(f, flag, CreateOpts.blockSize(1_000_000));\n\n// after\nlong bpc = 512;\nlong bs = ((1_000_000 + bpc - 1) / bpc) * bpc; // 1000448\nfc.create(f, flag, CreateOpts.blockSize(bs));","handlingStrategy":"validation","validationCode":"long bpc = CreateOpts.getOpt(CreateOpts.BytesPerChecksum.class, opts) != null\n    ? CreateOpts.getOpt(CreateOpts.BytesPerChecksum.class, opts).getValue() : 512;\nif (blockSize % bpc != 0) {\n  blockSize = ((blockSize + bpc - 1) / bpc) * bpc; // round up to a multiple\n}\nfc.create(f, flag, CreateOpts.blockSize(blockSize));","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Round computed block sizes to multiples of the checksum size before passing them","Omit blockSize and rely on server defaults unless tuning is required","When changing bytesPerChecksum, audit call sites that pass explicit block sizes"],"tags":["hadoop","file-create","blocksize","checksum","validation"],"backgroundTag":"blocksize-checksum-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}