apache/kafka · error · SchemaException

Error reading string of length ${length}, only ${remaining}

Error message

Error reading string of length ${length}, only ${remaining} bytes available

What it means

Thrown by stringRead() when the STRING field's declared length is positive and within the INT16 limit, but fewer bytes remain in the ByteBuffer than the length claims. This is the canonical truncated-frame / short-read guard for the legacy STRING type and is raised as a SchemaException before any UTF-8 decode is attempted.

Source

Thrown at clients/src/main/java/org/apache/kafka/common/protocol/types/Type.java:124

        /**
         * Documentation of the Type.
         *
         * @return details about valid values, representation
         */
        public abstract String documentation();

        @Override
        public String toString() {
            return typeName();
        }
    }

    public static String stringRead(ByteBuffer buffer, int length) {
        if (length > Short.MAX_VALUE)
            throw new SchemaException("String length " + length + " is larger than the maximum string length.");
        if (length > buffer.remaining())
            throw new SchemaException("Error reading string of length " + length + ", only " + buffer.remaining() + " bytes available");
        String result = Utils.utf8(buffer, length);
        buffer.position(buffer.position() + length);
        return result;
    }

    public static ByteBuffer bytesRead(ByteBuffer buffer, int size) {
        if (size > buffer.remaining())
            throw new SchemaException("Error reading bytes of size " + size + ", only " + buffer.remaining() + " bytes available");

        int limit = buffer.limit();
        int newPosition = buffer.position() + size;
        buffer.limit(newPosition);
        ByteBuffer val = buffer.slice();
        buffer.limit(limit);
        buffer.position(newPosition);
        return val;
    }

View on GitHub (pinned to c31c9215e1)

Solutions

1. Ensure the full Kafka response frame is accumulated before deserialization (NetworkReceive / size-prefixed framing must complete).
2. Inspect buffer.position(), limit(), and remaining() at the failure site; log the declared length vs remaining to confirm truncation.
3. Verify no intermediary (proxy, TLS terminator, custom interceptor) is truncating or rewriting the payload.
4. Match client version to broker version for the failing API key to rule out a schema drift.

Example fix

// before: decoding a possibly-incomplete frame
String v = Type.stringRead(buffer, length); // throws when remaining < length

// after: only decode once the full frame is buffered
if (buffer.remaining() < length) {
    // accumulate more bytes from the socket / re-read before decoding
    return null;
}
String v = Utils.utf8(buffer, length);

// Both length and buffer.remaining() are known before the call.
if (length > buffer.remaining()) {
    throw new IllegalArgumentException(
        "Need " + length + " bytes but only " + buffer.remaining() + " available");
}
String value = org.apache.kafka.common.protocol.types.Type.stringRead(buffer, length);

When it happens

Trigger: STRING.read(buffer) (or stringRead via COMPACT_STRING.read) where buffer.remaining() < length after the length prefix was consumed. Reached during ApiMessage/Struct deserialization when the source buffer was sliced too short, the network read returned a partial frame, or a preceding field's length was wrong so the position is off.

Common situations: Non-blocking socket returned an incomplete response that was passed to the decoder anyway. A buffer was rewound/sliced incorrectly between fields. Client and broker disagree on the message schema (e.g. custom plugin or older client). Frame corruption on the wire from a proxy or load balancer.

Related errors

• Buffer underflow while parsing consumer protocol's header
• String length ${length} is larger than the maximum string le
• Error reading bytes of size ${size}, only ${remaining} bytes
• Malformed consumer protocol subscription
• Malformed consumer protocol assignment

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/76f5f34613250de9.json. Report an issue: GitHub.