{"record":{"id":"6afe0c4f9d523b0a","repo":"apache/hadoop","slug":"invalid-checksum-type-useropt-default-eff","errorCode":null,"errorMessage":"Invalid checksum type: userOpt={}, default={}, effective=null","messagePattern":"Invalid checksum type: userOpt=(.+?), default=(.+?), effective=null","errorType":"validation","errorClass":"HadoopIllegalArgumentException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DebugAdmin.java","lineNumber":262,"sourceCode":"          \"  Compute HDFS metadata from the specified block file, and save it\"\n              + \" to\" + System.lineSeparator()\n              + \"  the specified output metadata file.\"\n              + System.lineSeparator() + System.lineSeparator()\n              + \"**NOTE: Use at your own risk!\" + System.lineSeparator()\n              + \" If the block file is corrupt\"\n              + \" and you overwrite it's meta file, \" + System.lineSeparator()\n              + \" it will show up\"\n              + \" as good in HDFS, but you can't read the data.\"\n              + System.lineSeparator()\n              + \" Only use as a last measure, and when you are 100% certain\"\n              + \" the block file is good.\");\n    }\n\n    private DataChecksum createChecksum(Options.ChecksumOpt opt) {\n      DataChecksum dataChecksum = DataChecksum\n          .newDataChecksum(opt.getChecksumType(), opt.getBytesPerChecksum());\n      if (dataChecksum == null) {\n        throw new HadoopIllegalArgumentException(\n            \"Invalid checksum type: userOpt=\" + opt + \", default=\" + opt\n                + \", effective=null\");\n      }\n      return dataChecksum;\n    }\n\n    int run(List<String> args) throws IOException {\n      if (args.size() == 0) {\n        System.out.println(usageText);\n        System.out.println(helpText + System.lineSeparator());\n        return 1;\n      }\n      final String name = StringUtils.popOptionWithArgument(\"-block\", args);\n      if (name == null) {\n        System.err.println(\"You must specify a block file with -block\");\n        return 2;\n      }\n      final File blockFile = new File(name);","sourceCodeStart":244,"sourceCodeEnd":280,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/tools/DebugAdmin.java#L244-L280","documentation":"'hdfs debug computeMeta' recomputes a block metadata file and needs a DataChecksum matching the client settings. It reads the effective ChecksumOpt via DfsClientConf.getChecksumOptFromConf(conf) (dfs.checksum.type, dfs.bytes-per-checksum) and calls DataChecksum.newDataChecksum(type, bytesPerChecksum). That factory returns null — instead of throwing — when bytesPerChecksum <= 0 or the type is not creatable (only NULL, CRC32 and CRC32C can construct a checksum; DEFAULT and MIXED are markers). DebugAdmin converts the null into this HadoopIllegalArgumentException. The message text has a known formatting quirk: it prints the same option twice and always says 'effective=null'.","triggerScenarios":"Running 'hdfs debug computeMeta -block <file> -out <meta>' with dfs.bytes-per-checksum set to 0 or negative, or dfs.checksum.type resolving to a non-creatable type value (DEFAULT/MIXED) in the loaded configuration.","commonSituations":"A core-site.xml/hdfs-site.xml on the admin box overrides dfs.checksum.type or dfs.bytes-per-checksum with an invalid value for other tooling; debugging on a host with leftover experimental checksum config; NULL checksum expected but typo'd.","solutions":["Set dfs.checksum.type to a creatable value: crc32, crc32c, or null (case-insensitive).","Ensure dfs.bytes-per-checksum is a positive integer (default 512).","Run computeMeta with an explicit clean config: 'hdfs --config <clean-conf-dir> debug computeMeta ...' to bypass polluted settings.","Confirm the block file's real checksum type from its .meta header before overwriting metadata, since computeMeta trusts config, not the block."],"exampleFix":"# before: bad checksum settings in the active config\nhdfs debug computeMeta -block blk_1073741825 -out /tmp/blk_1073741825.meta\n# -> Invalid checksum type ... effective=null\n\n# after: force valid values\nhdfs dfsadmin -fs hdfs://nn -setChecksumType crc32c 2>/dev/null; \\\n  hdfs --config /etc/hadoop-clean debug computeMeta \\\n  -block blk_1073741825 -out /tmp/blk_1073741825.meta\n\n# or in core-site.xml\n<property><name>dfs.checksum.type</name><value>crc32c</value></property>\n<property><name>dfs.bytes-per-checksum</name><value>512</value></property>","handlingStrategy":"validation","validationCode":"Configuration conf = new Configuration();\nString type = conf.get(DFSConfigKeys.DFS_CHECKSUM_TYPE_KEY, \"crc32c\");\nint bpc = conf.getInt(DFSConfigKeys.DFS_BYTES_PER_CHECKSUM_KEY,\n    DFSConfigKeys.DFS_BYTES_PER_CHECKSUM_DEFAULT);\nboolean creatable = bpc > 0\n    && (\"crc32\".equalsIgnoreCase(type) || \"crc32c\".equalsIgnoreCase(type)\n        || \"null\".equalsIgnoreCase(type));\nif (!creatable) {\n  throw new IllegalArgumentException(\n      \"dfs.checksum.type=\" + type + \" / dfs.bytes-per-checksum=\" + bpc\n          + \" cannot build a DataChecksum for computeMeta\");\n}","typeGuard":null,"tryCatchPattern":"try {\n  debugAdmin.run(new String[]{\"computeMeta\", \"-block\", blk, \"-out\", out});\n} catch (HadoopIllegalArgumentException e) {\n  if (e.getMessage().startsWith(\"Invalid checksum type\")) {\n    // fix dfs.checksum.type / dfs.bytes-per-checksum in the active config, then rerun\n    System.err.println(\"Fix dfs.checksum.type (crc32|crc32c|null) and a positive \"\n        + \"dfs.bytes-per-checksum, then retry\");\n    return;\n  }\n  throw e;\n}","preventionTips":["Validate checksum keys in config linters: dfs.checksum.type must be crc32/crc32c/null and dfs.bytes-per-checksum > 0.","Run debug tools with an explicit --config directory pinned to known-good files.","Read the existing .meta header to learn the real checksum settings instead of trusting client config."],"tags":["hadoop","hdfs","checksum","debug-tool","configuration"],"backgroundTag":"unsupported-checksum-type","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}