apache/kafka · error · IllegalArgumentException

Headers cannot be null

Error message

Headers cannot be null

What it means

DefaultRecord.writeTo (line 200), and the matching sizeOfBodyInBytes/sizeOf helpers, reject a null Header[] because the v2 record format always serialises a header-count varint. The contract is 'no headers' = Record.EMPTY_HEADERS, never null; passing null indicates the caller failed to normalise.

Source

Thrown at clients/src/main/java/org/apache/kafka/common/record/internal/DefaultRecord.java:200

        if (key == null) {
            ByteUtils.writeVarint(-1, out);
        } else {
            int keySize = key.remaining();
            ByteUtils.writeVarint(keySize, out);
            Utils.writeTo(out, key, keySize);
        }

        if (value == null) {
            ByteUtils.writeVarint(-1, out);
        } else {
            int valueSize = value.remaining();
            ByteUtils.writeVarint(valueSize, out);
            Utils.writeTo(out, value, valueSize);
        }

        if (headers == null)
            throw new IllegalArgumentException("Headers cannot be null");

        ByteUtils.writeVarint(headers.length, out);

        for (Header header : headers) {
            String headerKey = header.key();
            if (headerKey == null)
                throw new IllegalArgumentException("Invalid null header key found in headers");

            byte[] utf8Bytes = Utils.utf8(headerKey);
            ByteUtils.writeVarint(utf8Bytes.length, out);
            out.write(utf8Bytes);

            byte[] headerValue = header.value();
            if (headerValue == null) {
                ByteUtils.writeVarint(-1, out);
            } else {
                ByteUtils.writeVarint(headerValue.length, out);
                out.write(headerValue);

View on GitHub (pinned to c31c9215e1)

Solutions

1. Pass Record.EMPTY_HEADERS (or new Header[0]) when there are no headers.
2. Initialise header fields eagerly at construction; never leave them null.
3. Normalise at the boundary: coerce null to an empty array before record assembly.

Example fix

// before
DefaultRecord.writeTo(out, delta, tsDelta, key, value, null);
// after
DefaultRecord.writeTo(out, delta, tsDelta, key, value, Record.EMPTY_HEADERS);

// Never pass null headers; substitute the empty array before producing/serializing
import org.apache.kafka.common.header.Header;
import org.apache.kafka.common.record.Record;

Header[] safeHeaders = (headers == null) ? Record.EMPTY_HEADERS : headers;
// now safe to pass to DefaultRecord.writeTo(...) or ProducerRecord

import org.apache.kafka.common.header.Header;
import org.apache.kafka.common.record.Record;

static Header[] nonNullHeaders(Header[] headers) {
    return headers == null ? Record.EMPTY_HEADERS : headers;
}

// usage: DefaultRecord.writeTo(out, offsetDelta, tsDelta, key, value, nonNullHeaders(headers));

try {
    DefaultRecord.writeTo(out, offsetDelta, tsDelta, key, value, headers);
} catch (IllegalArgumentException e) {
    // headers was null; replace with Record.EMPTY_HEADERS and retry
}

When it happens

Trigger: Constructing records via low-level paths (DefaultRecord.writeTo / sizeInBytes) with a null headers argument; custom serializers, interceptors, or converters that propagate null instead of an empty array.

Common situations: Libraries that wrap records and forget to default headers; bridging from a format that omits headers; tests hand-building records.

Related errors

• Invalid null header key found in headers
• Found invalid number of record headers {}
• Found invalid number of record headers. {} is larger than th
• Invalid negative header key size {}
• Headers cannot be null

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/368bcc71cac39b75.json. Report an issue: GitHub.