{"record":{"id":"ecaf32dfb3809878","repo":"apache/hadoop","slug":"a-record-version-mismatch-occurred-expecting-v","errorCode":null,"errorMessage":"A record version mismatch occurred. Expecting v{}, found v{}","messagePattern":"A record version mismatch occurred\\. Expecting v(.+?), found v(.+?)","errorType":"exception","errorClass":"VersionMismatchException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SequenceFile.java","lineNumber":1143,"sourceCode":"        Progressable progress = progressOption == null ? null :\n          progressOption.getValue();\n\n        if (appendIfExistsOption != null && appendIfExistsOption.getValue()\n            && fs.exists(p)) {\n\n          // Read the file and verify header details\n          SequenceFile.Reader reader = new SequenceFile.Reader(conf,\n              SequenceFile.Reader.file(p), new Reader.OnlyHeaderOption());\n          try {\n\n            if (keyClassOption.getValue() != reader.getKeyClass()\n                || valueClassOption.getValue() != reader.getValueClass()) {\n              throw new IllegalArgumentException(\n                  \"Key/value class provided does not match the file\");\n            }\n\n            if (reader.getVersion() != VERSION[3]) {\n              throw new VersionMismatchException(VERSION[3],\n                  reader.getVersion());\n            }\n\n            if (metadataOption != null) {\n              LOG.info(\"MetaData Option is ignored during append\");\n            }\n            metadataOption = (MetadataOption) SequenceFile.Writer\n                .metadata(reader.getMetadata());\n\n            CompressionOption readerCompressionOption = new CompressionOption(\n                reader.getCompressionType(), reader.getCompressionCodec());\n\n            // Codec comparison will be ignored if the compression is NONE\n            if (readerCompressionOption.value != compressionTypeOption.value\n                || (readerCompressionOption.value != CompressionType.NONE\n                    && readerCompressionOption.codec\n                        .getClass() != compressionTypeOption.codec\n                            .getClass())) {","sourceCodeStart":1125,"sourceCodeEnd":1161,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/io/SequenceFile.java#L1125-L1161","documentation":"In appendIfExists mode the existing file must have been written with sequence-file version VERSION[3] (the append-capable header revision); the writer opens a Reader, compares reader.getVersion() against it and throws VersionMismatchException (expected vs found version bytes) for anything older. Old headers lack the layout guarantees appending relies on, so the writer refuses rather than corrupt them.","triggerScenarios":"Appending (Writer.appendIfExists(true)) to a sequence file produced by an older Hadoop writer with a pre-append version byte, or to a file whose header was hand-edited/damaged so the version byte reads as something unexpected.","commonSituations":"Appending to data files archived from an old cluster; mixed-version pipelines where an old writer produced the file; tutorials/data generated years earlier being reused as append targets.","solutions":["Read the old file with SequenceFile.Reader and copy its records into a new file created by the current writer, then append to that file","Or simply write a new output part-file instead of appending to the legacy one","Standardize writer and reader on the same Hadoop version so headers are uniform"],"exampleFix":"// before\nWriter w = SequenceFile.createWriter(conf, Writer.file(oldFile), Writer.appendIfExists(true), ...); // VersionMismatchException\n\n// after: rewrite to a fresh file with the current writer, then append\ntry (SequenceFile.Reader in = new SequenceFile.Reader(conf, Reader.file(oldFile));\n     SequenceFile.Writer out = SequenceFile.createWriter(conf, Writer.file(newFile),\n         Writer.keyClass(in.getKeyClass()), Writer.valueClass(in.getValueClass()))) {\n  Object k = ReflectionUtils.newInstance(in.getKeyClass(), conf);\n  Object v = ReflectionUtils.newInstance(in.getValueClass(), conf);\n  while (in.next(k, v)) { out.append(k, v); }\n}","handlingStrategy":"validation","validationCode":"// sequence file header: 3 magic bytes + 1 version byte; append needs version 6\ntry (FSDataInputStream in = fs.open(p)) {\n  byte[] hdr = new byte[4];\n  in.readFully(hdr);\n  if (hdr[3] != 6) {\n    throw new IOException(\"file version \" + hdr[3] + \" does not support append; rewrite it first\");\n  }\n}","typeGuard":null,"tryCatchPattern":"try {\n  w = SequenceFile.createWriter(conf, opts); // appendIfExists(true)\n} catch (VersionMismatchException e) {\n  // expected vs found version in the message; rewrite the file with the current writer\n}","preventionTips":["Re-archive legacy sequence files through a current-version copy job before using them as append targets","Prefer writing new part-files over appending to historical data"],"tags":["hadoop","sequence-file","append","version-mismatch"],"backgroundTag":"file-version-mismatch","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}