{"record":{"id":"b3b29db6cb308b5b","repo":"apache/hadoop","slug":"internal-error-default-blocksize-is-not-a-multipl","errorCode":null,"errorMessage":"Internal error: default blockSize is not a multiple of default bytesPerChecksum ","messagePattern":"Internal error: default blockSize is not a multiple of default bytesPerChecksum ","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java","lineNumber":606,"sourceCode":"      } else if (CreateOpts.CreateParent.class.isInstance(iOpt)) {\n        if (createParent != null) {\n          throw new HadoopIllegalArgumentException(\n              \"CreateParent option is set multiple times\");\n        }\n        createParent = ((CreateOpts.CreateParent) iOpt).getValue();\n      } else {\n        throw new HadoopIllegalArgumentException(\"Unkown CreateOpts of type \" +\n            iOpt.getClass().getName());\n      }\n    }\n    if (permission == null) {\n      throw new HadoopIllegalArgumentException(\"no permission supplied\");\n    }\n\n\n    FsServerDefaults ssDef = getServerDefaults(f);\n    if (ssDef.getBlockSize() % ssDef.getBytesPerChecksum() != 0) {\n      throw new IOException(\"Internal error: default blockSize is\" + \n          \" not a multiple of default bytesPerChecksum \");\n    }\n    \n    if (blockSize == -1) {\n      blockSize = ssDef.getBlockSize();\n    }\n\n    // Create a checksum option honoring user input as much as possible.\n    // If bytesPerChecksum is specified, it will override the one set in\n    // checksumOpt. Any missing value will be filled in using the default.\n    ChecksumOpt defaultOpt = new ChecksumOpt(\n        ssDef.getChecksumType(),\n        ssDef.getBytesPerChecksum());\n    checksumOpt = ChecksumOpt.processChecksumOpt(defaultOpt,\n        checksumOpt, bytesPerChecksum);\n\n    if (bufferSize == -1) {\n      bufferSize = ssDef.getFileBufferSize();","sourceCodeStart":588,"sourceCodeEnd":624,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/fs/AbstractFileSystem.java#L588-L624","documentation":"Before resolving defaults, AbstractFileSystem.create fetches FsServerDefaults via getServerDefaults(f) and asserts server blockSize % bytesPerChecksum == 0; failure raises IOException(\"Internal error: ...\"). It signals that the effective server-side configuration (or a custom file system's FsServerDefaults) is self-inconsistent — the message explicitly calls it an internal error, not a user-input error.","triggerScenarios":"dfs.block.size configured to a value not divisible by io.bytes.per.checksum (default 512); a custom AbstractFileSystem/FileSystem returning a hand-built FsServerDefaults with inconsistent values; test stubs/fake file systems with arbitrary defaults.","commonSituations":"Operators tuning dfs.block.size to a round decimal (e.g. 1000000) that is not a multiple of 512; in-memory/minicluster setups with locally overridden defaults; third-party filesystem implementations with sloppy FsServerDefaults.","solutions":["Set dfs.block.size to a multiple of io.bytes.per.checksum (e.g. 134217728 = 128MB with 512-byte checksums)","Verify what getServerDefaults(path) actually returns for the target path/cluster and fix the source of the bad value","For custom file systems, build FsServerDefaults so blockSize is divisible by bytesPerChecksum (and sanity-check in tests)"],"exampleFix":"# before (core-site.xml / hdfs-site.xml)\n<property><name>dfs.block.size</name><value>1000000</value></property>\n\n# after\n<property><name>dfs.block.size</name><value>134217728</value></property>","handlingStrategy":"validation","validationCode":"FsServerDefaults d = afs.getServerDefaults(f);\nif (d.getBlockSize() % d.getBytesPerChecksum() != 0) {\n  throw new IllegalStateException(\"Server defaults inconsistent: blockSize=\" + d.getBlockSize()\n      + \" bytesPerChecksum=\" + d.getBytesPerChecksum() + \" — fix dfs.block.size / io.bytes.per.checksum\");\n}","typeGuard":null,"tryCatchPattern":"catch (IOException e) { if (e.getMessage().contains(\"not a multiple of default bytesPerChecksum\")) { /* config error: surface to operator, do not retry */ } else throw e; }","preventionTips":["Keep dfs.block.size a power-of-two multiple of io.bytes.per.checksum (default 512)","For custom file systems, build FsServerDefaults from a consistent base and test defaults in CI","Log getServerDefaults() values when onboarding a new cluster or FS implementation"],"tags":["hadoop","file-create","blocksize","checksum","configuration"],"backgroundTag":"blocksize-checksum-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}