{"record":{"id":"c15ea003a5ffedd2","repo":"apache/hadoop","slug":"can-t-scan-a-pre-transactional-edit-log","errorCode":null,"errorMessage":"Can't scan a pre-transactional edit log.","messagePattern":"Can't scan a pre-transactional edit log\\.","errorType":"exception","errorClass":"java.io.IOException","httpStatus":null,"severity":"error","filePath":"hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java","lineNumber":5384,"sourceCode":"      FSEditLogOp op = cache.get(opCode);\n      if (op == null) {\n        throw new IOException(\"Read invalid opcode \" + opCode);\n      }\n      if (NameNodeLayoutVersion.supports(\n            LayoutVersion.Feature.STORED_TXIDS, logVersion)) {\n        op.setTransactionId(in.readLong());\n      } else {\n        op.setTransactionId(HdfsServerConstants.INVALID_TXID);\n      }\n      op.readFields(in, logVersion);\n      return op;\n    }\n\n    @Override\n    public long scanOp() throws IOException {\n      if (!NameNodeLayoutVersion.supports(\n          LayoutVersion.Feature.STORED_TXIDS, logVersion)) {\n        throw new IOException(\"Can't scan a pre-transactional edit log.\");\n      }\n      FSEditLogOp op = decodeOp();\n      return op == null ?\n          HdfsServerConstants.INVALID_TXID : op.getTransactionId();\n    }\n  }\n\n  public void outputToXml(ContentHandler contentHandler) throws SAXException {\n    contentHandler.startElement(\"\", \"\", \"RECORD\", new AttributesImpl());\n    XMLUtils.addSaxString(contentHandler, \"OPCODE\", opCode.toString());\n    contentHandler.startElement(\"\", \"\", \"DATA\", new AttributesImpl());\n    XMLUtils.addSaxString(contentHandler, \"TXID\", \"\" + txid);\n    toXml(contentHandler);\n    contentHandler.endElement(\"\", \"\", \"DATA\");\n    contentHandler.endElement(\"\", \"\", \"RECORD\");\n  }\n\n  protected abstract void toXml(ContentHandler contentHandler)","sourceCodeStart":5366,"sourceCodeEnd":5402,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java#L5366-L5402","documentation":"LegacyReader.scanOp() refuses to scan an edit log whose layout version predates LayoutVersion.Feature.STORED_TXIDS (introduced in Hadoop 0.21). Older formats do not persist a transaction id with each operation, so a txid-based scan cannot return meaningful ids and throws instead. decodeOp() still works on such logs (it substitutes INVALID_TXID); only the scan API is restricted.","triggerScenarios":"Calling EditLogInputStream.scanOp()/scanNextOp() consumers on a 0.20.x-format edits file; txid-scanning recovery tooling or edit-log scanners pointed at a pre-0.21 log; mixing legacy edits files into a directory a modern tool scans.","commonSituations":"Reviving archives from very old 0.20-era clusters; pointing recovery or audit tooling at historical edits; metadata restored from pre-0.21 backups into a modern deployment.","solutions":["Confirm the log's era: the first bytes of a binary edits file are the (negative) layout version — inspect with 'xxd old_edits | head -2'.","Use a decode-based reader instead of scan-based processing, e.g. 'hdfs oev -i old_edits -o /tmp/old.xml'.","Replay the legacy log with a matching-era Hadoop distribution to convert it into a modern namespace.","Keep pre-0.21 format files out of directories that modern scan tooling iterates."],"exampleFix":"# before: scan-based tooling fails\nhdfs namenode -recover   # IOException: Can't scan a pre-transactional edit log.\n\n# after: check version, then decode instead of scan\nxxd old_edits | head -2\nhdfs oev -i old_edits -o /tmp/old.xml","handlingStrategy":"validation","validationCode":"short logVersion;\ntry (DataInputStream in = new DataInputStream(\n        new BufferedInputStream(new FileInputStream(editsFile)))) {\n  logVersion = in.readShort();\n}\nboolean scannable = NameNodeLayoutVersion.supports(\n    LayoutVersion.Feature.STORED_TXIDS, logVersion);\nif (!scannable) {\n  // skip txid-scan tooling; use a decode-based reader instead\n}","typeGuard":null,"tryCatchPattern":"try {\n  long txid = stream.scanNextOp();\n} catch (IOException e) {\n  if (e.getMessage().contains(\"pre-transactional\")) {\n    // fall back to decodeOp()-based processing on this log\n  }\n}","preventionTips":["Tag archived edits files with the Hadoop version that wrote them.","Convert legacy logs to a modern layout before running recovery or scan tooling.","Keep pre-0.21 metadata out of production name directories."],"tags":["hdfs","edit-log","version-compatibility","recovery","java"],"backgroundTag":"unsupported-file-version","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-22T20:17:22.307Z"}