{"record":{"id":"af639fca5b37a3fc","repo":"apache/hadoop","slug":"input-files-cannot-be-merged-as-they-have-differen-af639f","errorCode":null,"errorMessage":"Input files cannot be merged as they have different Key class compared to specified comparator","messagePattern":"Input files cannot be merged as they have different Key class compared to specified comparator","errorType":"validation","errorClass":"HadoopIllegalArgumentException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/MapFile.java","lineNumber":1081,"sourceCode":"        Reader reader = new Reader(inMapFiles[i], conf);\n        if (keyClass == null || valueClass == null) {\n          keyClass = (Class<WritableComparable>) reader.getKeyClass();\n          valueClass = (Class<Writable>) reader.getValueClass();\n        } else if (keyClass != reader.getKeyClass()\n            || valueClass != reader.getValueClass()) {\n          throw new HadoopIllegalArgumentException(\n              \"Input files cannot be merged as they\"\n                  + \" have different Key and Value classes\");\n        }\n        inReaders[i] = reader;\n      }\n\n      if (comparator == null) {\n        Class<? extends WritableComparable> cls;\n        cls = keyClass.asSubclass(WritableComparable.class);\n        this.comparator = WritableComparator.get(cls, conf);\n      } else if (comparator.getKeyClass() != keyClass) {\n        throw new HadoopIllegalArgumentException(\n            \"Input files cannot be merged as they\"\n                + \" have different Key class compared to\"\n                + \" specified comparator\");\n      }\n\n      outWriter = new MapFile.Writer(conf, outMapFile,\n          MapFile.Writer.keyClass(keyClass),\n          MapFile.Writer.valueClass(valueClass));\n    }\n\n    /**\n     * Merge all input files to output map file.<br>\n     * 1. Read first key/value from all input files to keys/values array. <br>\n     * 2. Select the least key and corresponding value. <br>\n     * 3. Write the selected key and value to output file. <br>\n     * 4. Replace the already written key/value in keys/values arrays with the\n     * next key/value from the selected input <br>\n     * 5. Repeat step 2-4 till all keys are read. <br>","sourceCodeStart":1063,"sourceCodeEnd":1099,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/MapFile.java#L1063-L1099","documentation":"Thrown by MapFile.Merger.open(Path[], Path) when an explicit comparator was supplied whose key class does not identically match the key class of the input files (comparator.getKeyClass() != keyClass). The merger needs the comparator both to select the least key across inputs and to verify output ordering; a comparator bound to a different key type would compare apples to oranges and corrupt the merged map.","triggerScenarios":"Constructing MapFile.Merger(conf, comparator) where comparator is, say, a Text comparator while all inputs have LongWritable keys; reusing a comparator captured from another job's configuration; comparing custom writable subclasses where the comparator's getKeyClass() is the superclass while files store the subclass (identity == check fails).","commonSituations":"Refactoring key types without updating the comparator passed to the Merger; copy-pasted merge utility code with a hard-coded comparator; classloader subtleties where the same class name resolves to different Class instances.","solutions":["Pass null as the comparator — Merger then derives WritableComparator.get(keyClass) from the files themselves.","Or supply a comparator created via WritableComparator.get(actualKeyClass, conf) so its key class matches by construction.","If using a custom comparator, make sure its getKeyClass() returns the exact class stored in the inputs."],"exampleFix":"// before: comparator bound to Text but inputs hold LongWritable\nWritableComparator cmp = WritableComparator.get(Text.class, conf);\nnew MapFile.Merger(conf, cmp).merge(inputs, false, out); // throws\n\n// after: derive the comparator from the files' actual key class\nClass<? extends WritableComparable> kc =\n    new MapFile.Reader(inputs[0], conf).getKeyClass().asSubclass(WritableComparable.class);\nnew MapFile.Merger(conf, WritableComparator.get(kc, conf)).merge(inputs, false, out);","handlingStrategy":"validation","validationCode":"if (comparator != null && comparator.getKeyClass() != keyClass) {\n  comparator = WritableComparator.get(keyClass.asSubclass(WritableComparable.class), conf);\n}\nnew MapFile.Merger(conf, comparator).merge(inputs, false, out);","typeGuard":null,"tryCatchPattern":"try {\n  new MapFile.Merger(conf, comparator).merge(inputs, deleteInputs, out);\n} catch (HadoopIllegalArgumentException e) {\n  if (e.getMessage().contains(\"specified comparator\")) {\n    // simplest recovery: pass null so Merger derives the comparator from inputs\n    new MapFile.Merger(conf, null).merge(inputs, deleteInputs, out);\n  } else { throw e; }\n}","preventionTips":["Default to passing null comparator — MapFile.Merger derives the right one from the input files' key class.","If a custom comparator is required, construct it via WritableComparator.get(actualKeyClass, conf) so key classes match by construction.","When key types are refactored, grep for every Comparator handoff into Merger/Writer/Reader and update in the same commit."],"tags":["mapfile","merge","comparator","type-mismatch","hadoop-common"],"backgroundTag":"incompatible-input-types","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}