{"id":"c2048a077099d112","repo":"apache/kafka","slug":"this-array-is-not-nullable","errorCode":null,"errorMessage":"This array is not nullable.","messagePattern":"This array is not nullable\\.","errorType":"exception","errorClass":"SchemaException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/common/protocol/types/CompactArrayOf.java","lineNumber":78,"sourceCode":"            ByteUtils.writeUnsignedVarint(0, buffer);\n            return;\n        }\n        Object[] objs = (Object[]) o;\n        int size = objs.length;\n        ByteUtils.writeUnsignedVarint(size + 1, buffer);\n\n        for (Object obj : objs)\n            type.write(buffer, obj);\n    }\n\n    @Override\n    public Object read(ByteBuffer buffer) {\n        int n = ByteUtils.readUnsignedVarint(buffer);\n        if (n == 0) {\n            if (isNullable()) {\n                return null;\n            } else {\n                throw new SchemaException(\"This array is not nullable.\");\n            }\n        }\n        int size = n - 1;\n        if (size > buffer.remaining())\n            throw new SchemaException(\"Error reading array of size \" + size + \", only \" + buffer.remaining() + \" bytes available\");\n        Object[] objs = new Object[size];\n        for (int i = 0; i < size; i++)\n            objs[i] = type.read(buffer);\n        return objs;\n    }\n\n    @Override\n    public int sizeOf(Object o) {\n        if (o == null) {\n            return 1;\n        }\n        Object[] objs = (Object[]) o;\n        int size = ByteUtils.sizeOfUnsignedVarint(objs.length + 1);","sourceCodeStart":60,"sourceCodeEnd":96,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/common/protocol/types/CompactArrayOf.java#L60-L96","documentation":"Thrown by CompactArrayOf.read when the varint length prefix decodes to 0 (the compact-array null marker) but the field was constructed as non-nullable via `new CompactArrayOf(type)`. Compact arrays encode null as 0 and N elements as N+1, so 0 is reserved exclusively for null on nullable arrays. A non-nullable array receiving 0 is a protocol contract violation.","triggerScenarios":"CompactArrayOf.read reads an unsigned varint N; when N == 0 and isNullable() is false the exception fires. Triggered by a writer that emitted a null compact array for a field the reader's schema marks non-nullable, or by buffer corruption that lands a 0 byte at the length position.","commonSituations":"Mismatched flexible-version schemas (tagged fields / compact arrays were introduced with KIP-482 flexible versions); a producer on a newer client writing null where an older broker's schema does not permit it; hand-built test payloads using 0 as a placeholder length.","solutions":["Declare the field as nullable: CompactArrayOf.nullable(type) instead of new CompactArrayOf(type).","Make producer and consumer use the same message version; flexible/v3+ APIs use compact arrays and must agree on nullability.","Inspect the byte at the buffer position; if it is not legitimately 0, trace where the buffer content originated (wrong schema version, custom serializer).","Regenerate any hand-maintained schemas from the message JSON specs via ./gradlew processMessages."],"exampleFix":"// before\nnew Schema(new Field(\"hosts\", new CompactArrayOf(STRING)))\n\n// after\nnew Schema(new Field(\"hosts\", CompactArrayOf.nullable(STRING)))","handlingStrategy":"try-catch","validationCode":"// Schema-definition check: if YOU construct the CompactArrayOf, decide\n// nullability up front. CompactArrayOf.read throws when a 0 varint arrives\n// on a non-nullable definition, which signals corrupt wire data to the reader.\nCompactArrayOf arrayType = ...; // your definition\nif (!arrayType.isNullable() && bytesMightEncodeEmptyAsZero) {\n    // either redefine as CompactArrayOf.nullable(...), or guarantee the writer\n    // never emits a 0 length for this field.\n}","typeGuard":null,"tryCatchPattern":"try {\n    Object[] arr = (Object[]) compactArrayOf.read(buffer);\n} catch (org.apache.kafka.common.protocol.types.SchemaException e) {\n    if (e.getMessage().equals(\"This array is not nullable.\")) {\n        // writer used the compact-nullable encoding (length 0 == null) but the\n        // reader schema is non-nullable -> schema/encoding mismatch.\n        log.error(\"Encoding mismatch on {}: nullable payload on non-nullable field\", field);\n        throw new IncompatibleSchemaException(e);\n    }\n    throw e;\n}","preventionTips":["Match CompactArrayOf nullability exactly between writer and reader; the compact encoding reserves 0 for null.","When migrating from ArrayOf to CompactArrayOf, audit nullability flags on every field.","Treat this message as a schema-incompatibility signal, not a transient I/O error; do not retry blindly.","Pin the protocol version negotiated with the peer so nullable vs non-nullable encodings agree."],"tags":["protocol","serialization","compact-array","flexible-versions","kafka-clients"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}