apache/kafka · error · RuntimeException

Reserved2 field must be 0

Error message

Reserved2 field must be 0

What it means

Thrown by BD.validate() when the low 4 bits (reserved2) of the LZ4 Frame Descriptor BD byte are non-zero. Per the LZ4 frame spec these bits are reserved and must be zero; BD only encodes the maximum block size in bits 4-6. A non-zero reserved2 indicates either a malformed frame or an unknown extension that this implementation refuses to read.

Source

Thrown at clients/src/main/java/org/apache/kafka/common/compress/Lz4BlockOutputStream.java:369

        private BD(int reserved2, int blockSizeValue, int reserved3) {
            this.reserved2 = reserved2;
            this.blockSizeValue = blockSizeValue;
            this.reserved3 = reserved3;
            validate();
        }

        public static BD fromByte(byte bd) {
            int reserved2 = (bd >>> 0) & 15;
            int blockMaximumSize = (bd >>> 4) & 7;
            int reserved3 = (bd >>> 7) & 1;

            return new BD(reserved2, blockMaximumSize, reserved3);
        }

        private void validate() {
            if (reserved2 != 0) {
                throw new RuntimeException("Reserved2 field must be 0");
            }
            if (blockSizeValue < 4 || blockSizeValue > 7) {
                throw new RuntimeException("Block size value must be between 4 and 7");
            }
            if (reserved3 != 0) {
                throw new RuntimeException("Reserved3 field must be 0");
            }
        }

        // 2^(2n+8)
        public int getBlockMaximumSize() {
            return 1 << ((2 * blockSizeValue) + 8);
        }

        public byte toByte() {
            return (byte) (((reserved2 & 15) << 0) | ((blockSizeValue & 7) << 4) | ((reserved3 & 1) << 7));
        }
    }

View on GitHub (pinned to c31c9215e1)

Solutions

  1. Confirm the producer uses Kafka's CompressionType.LZ4 (always emits reserved2=0); re-encode if it does not.
  2. Dump the bytes around the failing offset and confirm the parser is reading the BD byte at the correct position (after magic+FLG).
  3. If the data is external LZ4, decompress it with a general LZ4 library rather than feeding it to Lz4BlockInputStream.
  4. Re-fetch from another replica to rule out corruption of the BD byte in transit.

Example fix

// before: external producer sets reserved BD bits
//   BD byte = 0x1F  -> reserved2=0xF, blockMax=4 -> throws

// after: emit BD with reserved2=0 (Kafka default)
//   BD byte = 0x40  -> reserved2=0, blockMax=4 (64KB) -> valid
// Achieved by always constructing BD via the public constructor:
BD bd = new BD(BD.BLOCKSIZE_64KB); // reserved2 stays 0
Defensive patterns

Strategy: try-catch

Validate before calling

// BD.reserved2 is bits 0-3 of the BD byte; must be zero.
byte bdByte = frameBuf.get(bdOffset);
if ((bdByte & 0x0F) != 0) {
    throw new IOException("Refusing malformed LZ4 frame: BD reserved low nibble set (0x" + Integer.toHexString(bdByte & 0xff) + ")");
}

Type guard

null

Try / catch

try {
    try (Lz4BlockInputStream in = new Lz4BlockInputStream(payload, BufferSupplier.NO_CACHING, false)) {
        // consume
    }
} catch (RuntimeException e) {
    if (e.getMessage() != null && e.getMessage().contains("Reserved2 field must be 0")) {
        // Corrupt BD byte in LZ4 frame descriptor; treat as malformed record.
        log.warn("Malformed LZ4 BD byte (reserved2 set) on {}, skipping", tp);
        return;
    }
    throw e;
}

Prevention

When it happens

Trigger: BD.fromByte(b) where (b>>>0)&15 != 0, reached from readHeader() at line 128 while parsing the Frame Descriptor. Also thrown if a producer constructs BD with a non-zero reserved2 field (the public BD(int) constructor passes 0, so this only happens via the private constructor or by parsing external data).

Common situations: Consuming LZ4-compressed records produced by a non-Kafka encoder that sets reserved BD bits; bit-rot in the BD byte; misaligned read pointer causing the parser to read the wrong byte as BD; consuming data that has been re-encoded by an intermediary. Rare in pure Kafka deployments.

Related errors

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/040ce56bb712021e.json. Report an issue: GitHub.