apache/kafka · error · SchemaException

field size ${size} cannot be negative

Error message

field size ${size} cannot be negative

What it means

Thrown by TaggedFields.read when the per-field size varint decodes to a negative int. Although the size is read as an unsigned varint, an overlong/malformed varint can overflow into the sign bit and produce a negative int; the guard refuses it before sizing a byte[] allocation. It indicates corrupt bytes at the size position of a tagged field.

Source

Thrown at clients/src/main/java/org/apache/kafka/common/protocol/types/TaggedFields.java:97

    }

    @Override
    public NavigableMap<Integer, Object> read(ByteBuffer buffer) {
        int numTaggedFields = ByteUtils.readUnsignedVarint(buffer);
        if (numTaggedFields == 0) {
            return Collections.emptyNavigableMap();
        }
        NavigableMap<Integer, Object> objects = new TreeMap<>();
        int prevTag = -1;
        for (int i = 0; i < numTaggedFields; i++) {
            int tag = ByteUtils.readUnsignedVarint(buffer);
            if (tag <= prevTag) {
                throw new RuntimeException("Invalid or out-of-order tag " + tag);
            }
            prevTag = tag;
            int size = ByteUtils.readUnsignedVarint(buffer);
            if (size < 0)
                throw new SchemaException("field size " + size + " cannot be negative");
            if (size > buffer.remaining())
                throw new SchemaException("Error reading field of size " + size + ", only " + buffer.remaining() + " bytes available");

            Field field = fields.get(tag);
            if (field == null) {
                byte[] bytes = new byte[size];
                buffer.get(bytes);
                objects.put(tag, new RawTaggedField(tag, bytes));
            } else {
                objects.put(tag, field.type.read(buffer));
            }
        }
        return objects;
    }

    @SuppressWarnings("unchecked")
    @Override
    public int sizeOf(Object o) {

View on GitHub (pinned to c31c9215e1)

Solutions

1. Confirm the buffer position is at the tagged-fields section - i.e. all preceding struct fields for this API version were consumed.
2. Verify the API version matches: tagged fields only exist in flexible (v3+) messages; do not attempt to read TaggedFields for non-flexible versions.
3. Hex-dump the bytes around the position to confirm a well-formed unsigned varint; reject the frame if it is malformed.
4. If reproducing from a recorded payload, re-record with current client and schema versions.

Example fix

// before - assuming every version has tagged fields
NavigableMap<Integer, Object> tags = taggedFields.read(buf);

// after - only read tagged fields for flexible versions
if (apiVersion >= apiVersions.flexibleVersion(apiKey)) {
    tags = taggedFields.read(buf);
} else {
    tags = Collections.emptyNavigableMap();
}

// The size is a varint decoded inside TaggedFields.read; a negative result
// implies a malformed varint the caller cannot pre-see. Only a minimum
// buffer check is feasible before the call.
if (buffer == null || buffer.remaining() < 1) {
    throw new IllegalArgumentException("buffer too small for tagged-field size varint");
}

try {
    NavigableMap<Integer, Object> tf = taggedFields.read(buffer);
} catch (org.apache.kafka.common.protocol.types.SchemaException e) {
    if (e.getMessage().contains("field size") && e.getMessage().contains("cannot be negative")) {
        // malformed varint -> corrupt wire data
        throw new CorruptFrameException(e);
    }
    throw e;
}

When it happens

Trigger: TaggedFields.read calls ByteUtils.readUnsignedVarint(buffer) for the size; the resulting int is checked `size < 0`. Hit when the bytes at that position do not form a valid unsigned varint (e.g. a 6-byte overlong encoding, garbage from a misaligned buffer, or a frame whose tag was consumed incorrectly leaving non-varint bytes).

Common situations: Buffer position misalignment after reading the wrong number of parent-struct fields; a truncated/garbled flexible-version payload; man-in-the-middle or on-disk corruption; reading a non-tagged payload with a schema that expects tagged fields (or vice versa).

Related errors

• Invalid or out-of-order tag ${tag}
• Error reading field of size ${size}, only ${remaining} bytes
• This array is not nullable.
• Error reading array of size ${size}, only ${remaining} bytes
• Buffer underflow while parsing consumer protocol's header

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/d27fbc6dc79d329a.json. Report an issue: GitHub.