{"record":{"id":"6768de30f0c60db6","repo":"apache/hadoop","slug":"key-value-class-provided-does-not-match-the-file","errorCode":null,"errorMessage":"Key/value class provided does not match the file","messagePattern":"Key/value class provided does not match the file","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SequenceFile.java","lineNumber":1138,"sourceCode":"        short replication = replicationOption == null ? \n          fs.getDefaultReplication(p) :\n          (short) replicationOption.getValue();\n        long blockSize = blockSizeOption == null ? fs.getDefaultBlockSize(p) :\n          blockSizeOption.getValue();\n        Progressable progress = progressOption == null ? null :\n          progressOption.getValue();\n\n        if (appendIfExistsOption != null && appendIfExistsOption.getValue()\n            && fs.exists(p)) {\n\n          // Read the file and verify header details\n          SequenceFile.Reader reader = new SequenceFile.Reader(conf,\n              SequenceFile.Reader.file(p), new Reader.OnlyHeaderOption());\n          try {\n\n            if (keyClassOption.getValue() != reader.getKeyClass()\n                || valueClassOption.getValue() != reader.getValueClass()) {\n              throw new IllegalArgumentException(\n                  \"Key/value class provided does not match the file\");\n            }\n\n            if (reader.getVersion() != VERSION[3]) {\n              throw new VersionMismatchException(VERSION[3],\n                  reader.getVersion());\n            }\n\n            if (metadataOption != null) {\n              LOG.info(\"MetaData Option is ignored during append\");\n            }\n            metadataOption = (MetadataOption) SequenceFile.Writer\n                .metadata(reader.getMetadata());\n\n            CompressionOption readerCompressionOption = new CompressionOption(\n                reader.getCompressionType(), reader.getCompressionCodec());\n\n            // Codec comparison will be ignored if the compression is NONE","sourceCodeStart":1120,"sourceCodeEnd":1156,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SequenceFile.java#L1120-L1156","documentation":"With Writer.appendIfExists(true), createWriter opens the existing sequence file's header and compares its stored key/value classes (by class identity) against Writer.keyClass(...)/Writer.valueClass(...). Any difference throws this IllegalArgumentException instead of appending records of a foreign type and producing an unreadable file.","triggerScenarios":"SequenceFile.createWriter(conf, Writer.file(existingSeqFile), Writer.appendIfExists(true), Writer.keyClass(A.class), Writer.valueClass(B.class)) where the header holds other classes; also triggered by passing a subclass where the header stores the superclass (comparison is !=, not isAssignableFrom).","commonSituations":"Types evolved between runs (Text keys changed to LongWritable) while output paths stayed fixed; multiple jobs with different schemas writing the same path; refactoring that replaced a Writable with its subclass.","solutions":["Inspect the file's real classes with SequenceFile.Reader.getKeyClass()/getValueClass() and pass exactly those to the writer","If the schema intentionally changed, write to a new path instead of appending","Rename or delete the old file if it is stale and start a fresh one"],"exampleFix":"// before\nWriter w = SequenceFile.createWriter(conf, Writer.file(p), Writer.appendIfExists(true),\n    Writer.keyClass(LongWritable.class), Writer.valueClass(Text.class)); // file holds Text/IntWritable\n\n// after\ntry (SequenceFile.Reader probe = new SequenceFile.Reader(conf,\n    Reader.file(p), new Reader.OnlyHeaderOption())) {\n  Writer w = SequenceFile.createWriter(conf, Writer.file(p), Writer.appendIfExists(true),\n      Writer.keyClass(probe.getKeyClass()), Writer.valueClass(probe.getValueClass()));\n}","handlingStrategy":"validation","validationCode":"try (SequenceFile.Reader probe = new SequenceFile.Reader(conf,\n    SequenceFile.Reader.file(p), new SequenceFile.Reader.OnlyHeaderOption())) {\n  if (kc != probe.getKeyClass() || vc != probe.getValueClass()) {\n    throw new IllegalArgumentException(\"file holds \" + probe.getKeyClass() + \"/\" + probe.getValueClass()\n        + \" but caller passed \" + kc + \"/\" + vc);\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  w = SequenceFile.createWriter(conf, opts);\n} catch (IllegalArgumentException e) {\n  if (\"Key/value class provided does not match the file\".equals(e.getMessage())) {\n    // schema drift: write to a new path instead of appending\n  } else { throw e; }\n}","preventionTips":["When appending, always read the header first and reuse its key/value classes","Change output paths whenever the record schema changes"],"tags":["hadoop","sequence-file","append","schema-mismatch"],"backgroundTag":"schema-class-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}