apache/kafka · error · SchemaException

Array size ${size} cannot be negative

Error message

Array size ${size} cannot be negative

What it means

Thrown by ArrayOf.read when the INT32 length prefix read from the buffer is negative on a non-nullable array. In Kafka's protocol, only nullable arrays may use the length value -1 to encode null; a non-nullable array receiving a negative length is malformed. This guard rejects corrupt or misrouted bytes before they are used to size an allocation.

Source

Thrown at clients/src/main/java/org/apache/kafka/common/protocol/types/ArrayOf.java:75

            buffer.putInt(-1);
            return;
        }

        Object[] objs = (Object[]) o;
        int size = objs.length;
        buffer.putInt(size);

        for (Object obj : objs)
            type.write(buffer, obj);
    }

    @Override
    public Object read(ByteBuffer buffer) {
        int size = buffer.getInt();
        if (size < 0 && isNullable())
            return null;
        else if (size < 0)
            throw new SchemaException("Array size " + size + " cannot be negative");

        if (size > buffer.remaining())
            throw new SchemaException("Error reading array of size " + size + ", only " + buffer.remaining() + " bytes available");
        Object[] objs = new Object[size];
        for (int i = 0; i < size; i++)
            objs[i] = type.read(buffer);
        return objs;
    }

    @Override
    public int sizeOf(Object o) {
        int size = 4;
        if (o == null)
            return size;

        Object[] objs = (Object[]) o;
        for (Object obj : objs)
            size += type.sizeOf(obj);

View on GitHub (pinned to c31c9215e1)

Solutions

1. Check whether the field should accept null and declare it with ArrayOf.nullable(type) instead of new ArrayOf(type).
2. Verify the ByteBuffer position/limit and that the bytes correspond to the Schema being read (compare API key + API version on the wire to the schema you selected).
3. If the bytes come from an external/recorded source, confirm the producer and consumer are on compatible client versions for the message version in use.
4. Hex-dump the region around the buffer position to confirm the INT32 length is what your schema expects.

Example fix

// before
new Schema(new Field("topics", new ArrayOf(STRING)))

// after - allow null for an optional topic list
new Schema(new Field("topics", ArrayOf.nullable(STRING)))

// The negative size is decoded INSIDE ArrayOf.read, so the caller cannot
// inspect it beforehand. Only a cheap structural pre-check is possible:
if (buffer == null || buffer.remaining() < Integer.BYTES) {
    throw new IllegalArgumentException("buffer too small for array length prefix");
}
// Call site still must tolerate a corrupt negative length from the wire:
schemaType.read(buffer);

try {
    Object[] arr = (Object[]) arrayOf.read(buffer);
} catch (org.apache.kafka.common.protocol.types.SchemaException e) {
    // message starts with "Array size ... cannot be negative" -> corrupt/truncated payload
    // or client/broker API-version skew. Drop the payload, do not retry unchanged bytes.
    log.warn("Discarding malformed frame: {}", e.getMessage());
    throw new CorruptFrameException(e);
}

When it happens

Trigger: Calling Struct.read (or ArrayOf.read directly) on a ByteBuffer whose next INT32 is < 0 when the schema field was declared with `new ArrayOf(type)` rather than `ArrayOf.nullable(type)`. Also hit when a peer writes -1 as the array length for a field the local schema considers non-nullable.

Common situations: Broker/client version skew where one side treats an array as nullable and the other does not; a truncated or garbage ByteBuffer passed to the serializer (e.g. log corruption, wrong API key routing, misconfigured custom serialization); replaying a captured frame against the wrong Schema instance.

Related errors

• Error writing field '${name}': ${detail}
• Error reading field '${name}': ${detail}
• Buffer underflow while parsing consumer protocol's header
• Malformed consumer protocol subscription
• Malformed consumer protocol assignment

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/8e3ba24803f9855f.json. Report an issue: GitHub.