apache/hadoop · error · IOException

Unknown ChecksumCombineMode: {}

Error message

Unknown ChecksumCombineMode: {}

What it means

FileChecksumComputer's constructor switches on the ChecksumCombineMode that controls how per-block checksums are combined into a file checksum (dfs.checksum.combine.mode). Only MD5MD5CRC and COMPOSITE_CRC exist; any other value throws this defensive IOException at construction. In practice the config parser (DfsClientConf) already falls back to MD5MD5CRC on bad strings, so reaching this branch requires an enum value the running jar does not know — i.e., a ChecksumCombineMode constant introduced by newer code but absent here.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs-client/src/main/java/org/apache/hadoop/hdfs/FileChecksumHelper.java:116

                         LocatedBlocks blockLocations,
                         ClientProtocol namenode,
                         DFSClient client,
                         ChecksumCombineMode combineMode) throws IOException {
      this.src = src;
      this.length = length;
      this.blockLocations = blockLocations;
      this.namenode = namenode;
      this.client = client;
      this.combineMode = combineMode;
      switch (combineMode) {
      case MD5MD5CRC:
        this.blockChecksumType = BlockChecksumType.MD5CRC;
        break;
      case COMPOSITE_CRC:
        this.blockChecksumType = BlockChecksumType.COMPOSITE_CRC;
        break;
      default:
        throw new IOException("Unknown ChecksumCombineMode: " + combineMode);
      }

      this.remaining = length;

      if (blockLocations != null) {
        if (src.contains(HdfsConstants.SEPARATOR_DOT_SNAPSHOT_DIR_SEPARATOR)) {
          this.remaining = Math.min(length, blockLocations.getFileLength());
        }
        this.locatedBlocks = blockLocations.getLocatedBlocks();
      }
    }

    String getSrc() {
      return src;
    }

    long getLength() {
      return length;

View on GitHub (pinned to 2add963021)

Solutions

1. Set dfs.checksum.combine.mode to a known value: MD5MD5CRC (default) or COMPOSITE_CRC
2. Align all Hadoop jars (hadoop-hdfs-client, hadoop-common) to one version on the client classpath
3. If you run a fork that added a combine mode, make sure every node/client jar includes the new enum

Example fix

# before
dfs.checksum.combine.mode=COMPOSITE_CRC_EXTRA   # unknown to this client jar

# after
dfs.checksum.combine.mode=COMPOSITE_CRC          # or MD5MD5CRC

// Fail fast at startup instead of at first checksum call:
String mode = conf.get("dfs.checksum.combine.mode", "MD5MD5CRC");
if (!"MD5MD5CRC".equals(mode) && !"COMPOSITE_CRC".equals(mode)) {
  throw new IllegalArgumentException("Invalid dfs.checksum.combine.mode: " + mode);
}

try {
  checksum = dfsClient.getFileChecksum(path);
} catch (IOException e) {
  if (e.getMessage().contains("Unknown ChecksumCombineMode")) {
    // classpath/version skew: log and retry with explicit default mode
    conf.set("dfs.checksum.combine.mode", "MD5MD5CRC");
  } else { throw e; }
}

When it happens

Trigger: DFSClient.getFileChecksum() invoked while DfsClientConf holds a ChecksumCombineMode other than MD5MD5CRC/COMPOSITE_CRC; a new combine-mode enum value serialized or injected from newer components; unit tests constructing FileChecksumComputer with a mock/unknown mode.

Common situations: Mixed Hadoop jar versions on one client classpath (newer hdfs-client jar providing a new enum value, older helper code); custom forks adding combine modes; typo attempts in dfs.checksum.combine.mode are normally intercepted earlier by the valueOf fallback, so a genuine hit points at classpath skew.

Related errors

• Unknown BlockChecksumType: {}
• Invalid checksum type in dfs.checksum.type: {}
• Invalid checksum type: userOpt={}, default={}, effective=nul
• Invalid values: dfs.bytes-per-checksum (={}) must divide cel
• Byte-per-checksum not matched: bpc={} but bytesPerCRC={}

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/5251af3c382ccc33. Report an issue: GitHub.