apache/hadoop · error · java.io.IOException

Op {opCodeByte} has size {opLength}, but the minimum op size

Error message

Op {opCodeByte} has size {opLength}, but the minimum op size is {MIN_OP_LENGTH}

What it means

Companion minimum-size check in decodeOpFrame(): a length-prefixed record needs at least MIN_OP_LENGTH = 17 bytes (1 opcode + 4 length + 8 txid + 0-byte body + 4 checksum, FSEditLogOp.java:5168). An opLength below 17 means the length field itself is corrupt or the segment is truncated mid-header, so the reader refuses to decode.

Source

Thrown at hadoop-hdfs-project/hadoop-hdfs/src/main/java/org/apache/hadoop/hdfs/server/namenode/FSEditLogOp.java:5252

      } catch (EOFException eof) {
        // EOF at an opcode boundary is expected.
        return HdfsServerConstants.INVALID_TXID;
      }
      if (opCodeByte == FSEditLogOpCodes.OP_INVALID.getOpCode()) {
        verifyTerminator();
        return HdfsServerConstants.INVALID_TXID;
      }
      // Here, we verify that the Op size makes sense and that the
      // data matches its checksum before attempting to construct an Op.
      // This is important because otherwise we may encounter an
      // OutOfMemoryException which could bring down the NameNode or
      // JournalNode when reading garbage data.
      int opLength =  in.readInt() + OP_ID_LENGTH + CHECKSUM_LENGTH;
      if (opLength > maxOpSize) {
        throw new IOException("Op " + (int)opCodeByte + " has size " +
            opLength + ", but maxOpSize = " + maxOpSize);
      } else  if (opLength < MIN_OP_LENGTH) {
        throw new IOException("Op " + (int)opCodeByte + " has size " +
            opLength + ", but the minimum op size is " + MIN_OP_LENGTH);
      }
      long txid = in.readLong();
      // Verify checksum
      in.reset();
      in.mark(maxOpSize);
      checksum.reset();
      for (int rem = opLength - CHECKSUM_LENGTH; rem > 0;) {
        int toRead = Math.min(temp.length, rem);
        IOUtils.readFully(in, temp, 0, toRead);
        checksum.update(temp, 0, toRead);
        rem -= toRead;
      }
      int expectedChecksum = in.readInt();
      int calculatedChecksum = (int)checksum.getValue();
      if (expectedChecksum != calculatedChecksum) {
        throw new ChecksumException(
            "Transaction is corrupt. Calculated checksum is " +

View on GitHub (pinned to 2add963021)

Solutions

Run 'hdfs namenode -recover' to skip the damaged tail and finish startup, then checkpoint
Verify where truncation begins using 'hdfs offlineEditsViewer'
Fetch an intact copy of the segment from healthy journals (QJM) or fall back to the last fsimage

Defensive patterns

Strategy: try-catch

Validate before calling

hdfs offlineEditsViewer -i <edits segment> -o /dev/null   # detects truncated/corrupt record headers pre-start

Try / catch

try {
  reader.readOp();
} catch (IOException e) {
  if (e.getMessage().contains("minimum op size")) {
    // truncated header or corrupt length field: recover-skip or restore clean copy
  } else { throw e; }
}

Prevention

Guard journal disks against fills; truncation happens at the tail
Validate segments after crashes before restart
Keep quorum journal copies so truncation never leaves zero good copies

When it happens

Trigger: The segment ends (truncation) inside a record header so the length int is partial; the length int is corrupt to a small/negative value; the stream is misaligned after earlier damage.

Common situations: Truncated tails from crash or disk-full; bit rot on journal storage; misparsed preceding record shifting offsets.

Related errors

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/b33706780261103f. Report an issue: GitHub.