apache/kafka · error · IllegalArgumentException

Invalid null header key found in headers

Error message

Invalid null header key found in headers

What it means

DefaultRecord.writeTo (line 207) and sizeOf reject any Header whose key() returns null. The wire format requires a length-prefixed UTF-8 key for every header, so a null key is a programmer error, not a valid 'no key' state.

Source

Thrown at clients/src/main/java/org/apache/kafka/common/record/internal/DefaultRecord.java:207

        }

        if (value == null) {
            ByteUtils.writeVarint(-1, out);
        } else {
            int valueSize = value.remaining();
            ByteUtils.writeVarint(valueSize, out);
            Utils.writeTo(out, value, valueSize);
        }

        if (headers == null)
            throw new IllegalArgumentException("Headers cannot be null");

        ByteUtils.writeVarint(headers.length, out);

        for (Header header : headers) {
            String headerKey = header.key();
            if (headerKey == null)
                throw new IllegalArgumentException("Invalid null header key found in headers");

            byte[] utf8Bytes = Utils.utf8(headerKey);
            ByteUtils.writeVarint(utf8Bytes.length, out);
            out.write(utf8Bytes);

            byte[] headerValue = header.value();
            if (headerValue == null) {
                ByteUtils.writeVarint(-1, out);
            } else {
                ByteUtils.writeVarint(headerValue.length, out);
                out.write(headerValue);
            }
        }

        return ByteUtils.sizeOfVarint(sizeInBytes) + sizeInBytes;
    }

    @Override

View on GitHub (pinned to c31c9215e1)

Solutions

1. Validate header keys before adding: reject null, or substitute an empty/default string.
2. Filter null-key entries out of the header map before record assembly.
3. If using RecordHeader, ensure the key argument is non-null at construction.

Example fix

// before
headers.add(new RecordHeader(mapKey, value));
// after
if (mapKey == null) throw new IllegalArgumentException("header key required");
headers.add(new RecordHeader(mapKey, value));

// Reject or drop records whose headers contain a null key before serializing
import org.apache.kafka.common.header.Header;
import org.apache.kafka.common.record.Record;

Header[] sanitized;
if (headers == null) {
    sanitized = Record.EMPTY_HEADERS;
} else {
    java.util.List<Header> kept = new java.util.ArrayList<>(headers.length);
    for (Header h : headers) {
        if (h != null && h.key() != null) kept.add(h);
    }
    sanitized = kept.toArray(new Header[0]);
}
// pass sanitized to DefaultRecord.writeTo / ProducerRecord

import org.apache.kafka.common.header.Header;

static boolean allHeaderKeysPresent(Header[] headers) {
    if (headers == null) return false;
    for (Header h : headers) {
        if (h == null || h.key() == null) return false;
    }
    return true;
}

// usage: if (allHeaderKeysPresent(headers)) { DefaultRecord.writeTo(...); }

try {
    DefaultRecord.writeTo(out, offsetDelta, tsDelta, key, value, headers);
} catch (IllegalArgumentException e) {
    // a header key was null; strip offending headers or fail the record
}

When it happens

Trigger: Constructing new RecordHeader(null, value) or a custom Header implementation whose key() returns null, then serialising the record; copying header keys from an untrusted map without null checks.

Common situations: Dynamic header builders fed by user input or maps; mocking headers in tests; converters that copy headers verbatim from a source allowing null keys.

Related errors

• Headers cannot be null
• Found invalid number of record headers {}
• Found invalid number of record headers. {} is larger than th
• Invalid negative header key size {}
• Headers cannot be null

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/ae9889e0fef2095e.json. Report an issue: GitHub.