{"id":"4c7d1462490e7630","repo":"apache/kafka","slug":"error-reading-array-of-size-size-only-remain","errorCode":null,"errorMessage":"Error reading array of size ${size}, only ${remaining} bytes available","messagePattern":"Error reading array of size (.+?), only (.+?) bytes available","errorType":"exception","errorClass":"SchemaException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/common/protocol/types/ArrayOf.java","lineNumber":78,"sourceCode":"\n        Object[] objs = (Object[]) o;\n        int size = objs.length;\n        buffer.putInt(size);\n\n        for (Object obj : objs)\n            type.write(buffer, obj);\n    }\n\n    @Override\n    public Object read(ByteBuffer buffer) {\n        int size = buffer.getInt();\n        if (size < 0 && isNullable())\n            return null;\n        else if (size < 0)\n            throw new SchemaException(\"Array size \" + size + \" cannot be negative\");\n\n        if (size > buffer.remaining())\n            throw new SchemaException(\"Error reading array of size \" + size + \", only \" + buffer.remaining() + \" bytes available\");\n        Object[] objs = new Object[size];\n        for (int i = 0; i < size; i++)\n            objs[i] = type.read(buffer);\n        return objs;\n    }\n\n    @Override\n    public int sizeOf(Object o) {\n        int size = 4;\n        if (o == null)\n            return size;\n\n        Object[] objs = (Object[]) o;\n        for (Object obj : objs)\n            size += type.sizeOf(obj);\n        return size;\n    }\n","sourceCodeStart":60,"sourceCodeEnd":96,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/common/protocol/types/ArrayOf.java#L60-L96","documentation":"Thrown by ArrayOf.read when the declared array length exceeds the bytes left in the ByteBuffer. It is a sanity guard preventing an oversized allocation and an underflow when element deserialization would run past the buffer end. The library treats this as unrecoverable wire corruption.","triggerScenarios":"ArrayOf.read reads a length N then checks `size > buffer.remaining()`; the next N elements of the element Type cannot fit. Happens when the element Type's sizeOf assumptions differ from what was written, or when the buffer was sliced/truncated before being passed to read.","commonSituations":"Message body was truncated by a network layer or truncated ByteBuffer slice; reading a payload with the wrong API version's schema (so the length prefix is actually some other field's bytes); corrupted on-disk log being replayed; a custom interceptor that re-slices buffers incorrectly.","solutions":["Confirm the ByteBuffer passed to read covers the complete message body (check limit-position vs. the framed size from the transport).","Ensure you are decoding with the Schema for the exact API key + API version present on the wire.","If reading from a log/record, verify the record offset and segment integrity (e.g. kafka-dump-log) before deserializing.","Add a length check in your framing layer so partial reads are rejected before reaching the schema."],"exampleFix":"// before - reading a possibly partial slice\nStruct s = (Struct) schema.read(slice);\n\n// after - validate the slice spans the whole framed message first\nif (slice.remaining() < framedSize) {\n    throw new IllegalStateException(\"underfull frame: \" + slice.remaining() + \" < \" + framedSize);\n}\nStruct s = (Struct) schema.read(slice);","handlingStrategy":"try-catch","validationCode":"// Best-effort: ensure the buffer at least carries the 4-byte length prefix.\n// The declared size is read internally, so a full pre-check is impossible;\n// pair this with the try-catch below.\nif (buffer == null || buffer.remaining() < Integer.BYTES) {\n    throw new IllegalArgumentException(\"buffer has no array-length prefix\");\n}","typeGuard":null,"tryCatchPattern":"try {\n    Object[] arr = (Object[]) arrayOf.read(buffer);\n} catch (org.apache.kafka.common.protocol.types.SchemaException e) {\n    if (e.getMessage().contains(\"only\") && e.getMessage().contains(\"bytes available\")) {\n        // truncated frame: size claimed more than the wire actually delivered\n        handleShortRead(e);\n    } else {\n        throw e;\n    }\n}","preventionTips":["Frame messages with an explicit length prefix at the transport layer so partial reads are detected before deserialization.","Treat a short buffer as a connection-level failure: close and re-establish rather than retrying the same bytes.","When slicing buffers for sub-readers, double-check slice length equals the documented payload size.","Log buffer.remaining() at the framing boundary to catch truncation early in diagnostics."],"tags":["protocol","serialization","buffer-underflow","kafka-clients"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}