What causes "This array is not nullable." in apache/kafka?

Declare the field as nullable: CompactArrayOf.nullable(type) instead of new CompactArrayOf(type).

apache/kafka · error · SchemaException

This array is not nullable.

Error message

This array is not nullable.

What it means

Thrown by CompactArrayOf.read when the varint length prefix decodes to 0 (the compact-array null marker) but the field was constructed as non-nullable via `new CompactArrayOf(type)`. Compact arrays encode null as 0 and N elements as N+1, so 0 is reserved exclusively for null on nullable arrays. A non-nullable array receiving 0 is a protocol contract violation.

Source

Thrown at clients/src/main/java/org/apache/kafka/common/protocol/types/CompactArrayOf.java:78

            ByteUtils.writeUnsignedVarint(0, buffer);
            return;
        }
        Object[] objs = (Object[]) o;
        int size = objs.length;
        ByteUtils.writeUnsignedVarint(size + 1, buffer);

        for (Object obj : objs)
            type.write(buffer, obj);
    }

    @Override
    public Object read(ByteBuffer buffer) {
        int n = ByteUtils.readUnsignedVarint(buffer);
        if (n == 0) {
            if (isNullable()) {
                return null;
            } else {
                throw new SchemaException("This array is not nullable.");
            }
        }
        int size = n - 1;
        if (size > buffer.remaining())
            throw new SchemaException("Error reading array of size " + size + ", only " + buffer.remaining() + " bytes available");
        Object[] objs = new Object[size];
        for (int i = 0; i < size; i++)
            objs[i] = type.read(buffer);
        return objs;
    }

    @Override
    public int sizeOf(Object o) {
        if (o == null) {
            return 1;
        }
        Object[] objs = (Object[]) o;
        int size = ByteUtils.sizeOfUnsignedVarint(objs.length + 1);

View on GitHub (pinned to c31c9215e1)

Solutions

Declare the field as nullable: CompactArrayOf.nullable(type) instead of new CompactArrayOf(type).
Make producer and consumer use the same message version; flexible/v3+ APIs use compact arrays and must agree on nullability.
Inspect the byte at the buffer position; if it is not legitimately 0, trace where the buffer content originated (wrong schema version, custom serializer).
Regenerate any hand-maintained schemas from the message JSON specs via ./gradlew processMessages.

Example fix

// before
new Schema(new Field("hosts", new CompactArrayOf(STRING)))

// after
new Schema(new Field("hosts", CompactArrayOf.nullable(STRING)))

Defensive patterns

Strategy: try-catch

Validate before calling

// Schema-definition check: if YOU construct the CompactArrayOf, decide
// nullability up front. CompactArrayOf.read throws when a 0 varint arrives
// on a non-nullable definition, which signals corrupt wire data to the reader.
CompactArrayOf arrayType = ...; // your definition
if (!arrayType.isNullable() && bytesMightEncodeEmptyAsZero) {
    // either redefine as CompactArrayOf.nullable(...), or guarantee the writer
    // never emits a 0 length for this field.
}

Try / catch

try {
    Object[] arr = (Object[]) compactArrayOf.read(buffer);
} catch (org.apache.kafka.common.protocol.types.SchemaException e) {
    if (e.getMessage().equals("This array is not nullable.")) {
        // writer used the compact-nullable encoding (length 0 == null) but the
        // reader schema is non-nullable -> schema/encoding mismatch.
        log.error("Encoding mismatch on {}: nullable payload on non-nullable field", field);
        throw new IncompatibleSchemaException(e);
    }
    throw e;
}

Prevention

Match CompactArrayOf nullability exactly between writer and reader; the compact encoding reserves 0 for null.
When migrating from ArrayOf to CompactArrayOf, audit nullability flags on every field.
Treat this message as a schema-incompatibility signal, not a transient I/O error; do not retry blindly.
Pin the protocol version negotiated with the peer so nullable vs non-nullable encodings agree.

When it happens

Trigger: CompactArrayOf.read reads an unsigned varint N; when N == 0 and isNullable() is false the exception fires. Triggered by a writer that emitted a null compact array for a field the reader's schema marks non-nullable, or by buffer corruption that lands a 0 byte at the length position.

Common situations: Mismatched flexible-version schemas (tagged fields / compact arrays were introduced with KIP-482 flexible versions); a producer on a newer client writing null where an older broker's schema does not permit it; hand-built test payloads using 0 as a placeholder length.

Related errors

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/c2048a077099d112.json. Report an issue: GitHub.