apache/hadoop · error · java.io.IOException

Read invalid opcode {opCode}

Error message

Read invalid opcode {opCode}

What it means

LengthPrefixedReader.decodeOp() (modern length+txid+checksum segments) maps the record's first byte to FSEditLogOpCodes and looks it up in the OpInstanceCache; a byte with no known opcode yields no cached op instance and this IOException. In effect: the stream position holds a byte that is not a valid opcode for this binary.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java:5203

    LengthPrefixedReader(DataInputStream in, StreamLimiter limiter,
                         int logVersion) {
      super(in, limiter, logVersion);
      this.checksum = DataChecksum.newCrc32();
    }

    @Override
    public FSEditLogOp decodeOp() throws IOException {
      long txid = decodeOpFrame();
      if (txid == HdfsServerConstants.INVALID_TXID) {
        return null;
      }
      in.reset();
      in.mark(maxOpSize);
      FSEditLogOpCodes opCode = FSEditLogOpCodes.fromByte(in.readByte());
      FSEditLogOp op = cache.get(opCode);
      if (op == null) {
        throw new IOException("Read invalid opcode " + opCode);
      }
      op.setTransactionId(txid);
      IOUtils.skipFully(in, 4 + 8); // skip length and txid
      op.readFields(in, logVersion);
      // skip over the checksum, which we validated above.
      IOUtils.skipFully(in, CHECKSUM_LENGTH);
      return op;
    }

    @Override
    public long scanOp() throws IOException {
      return decodeOpFrame();
    }

    /**
     * Decode the opcode "frame".  This includes reading the opcode and
     * transaction ID, and validating the checksum and length.  It does not
     * include reading the opcode-specific fields.

View on GitHub (pinned to 2add963021)

Solutions

1. If you downgraded, roll forward: restart with the newer release, run a checkpoint (saveNamespace), and only then downgrade again
2. Run 'hdfs namenode -recover' to skip the unknown record — accepting that this transaction is lost from replay
3. Confirm the segment's producing version with 'hdfs offlineEditsViewer -i <edits> -o out.xml' run from the release that wrote it
4. Replace the corrupt segment with a healthy journal copy if available

# before any downgrade, confirm no unknown-to-old-binary ops exist
hdfs offlineEditsViewer -i <newest edits segment> -o /dev/null   # run with the OLDER release

try {
  reader.readOp();
} catch (IOException e) {
  if (e.getMessage().contains("Read invalid opcode")) {
    // unknown opcode byte: version skew or corruption
    // roll forward to the newer release and checkpoint, or recover-skip
  } else { throw e; }
}

When it happens

Trigger: The stream is misaligned after earlier corruption so a body byte is read as an opcode; the file being read is not an edit log; or the segment was written by a newer Hadoop whose opcode values this older binary does not know (downgrade scenario).

Common situations: Downgrading NameNode software after newer ops were logged; tools pointed at a random/corrupt file in the current dir; single-byte corruption at a record boundary.

Related errors

• Invalid operation read {}
• The log file {} seems to contain valid transactions ; journa
• Same delegation token being added twice; invalid entry in fs
• No header found in log
• EOF while reading layout flags from log

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/e8857ec89cf482d2. Report an issue: GitHub.