apache/flink · error · KryoBufferUnderflowException

Buffer underflow.

Error message

Buffer underflow.

What it means

Thrown inside NoFetchingInput.require() when the underlying InputStream hits EOF (fill returns -1) while trying to top up the buffer to satisfy a read of 'required' bytes. It means the serialized stream ended before a complete value was available.

Source

Thrown at flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime/NoFetchingInput.java:83

        // The main change between this and Kryo 5 Input.require is this will never read more bytes
        // than required.
        // There are also formatting changes to be compliant with the Flink project styling rules.
        int remaining = limit - position;
        if (remaining >= required) {
            return remaining;
        }
        if (required > capacity) {
            throw new KryoException(
                    "Buffer too small: capacity: " + capacity + ", required: " + required);
        }

        int count;
        // Try to fill the buffer.
        if (remaining > 0) {
            // Logical change 1 (from Kryo Input.require): "capacity - limit" -> "required - limit"
            count = fill(buffer, limit, required - limit);
            if (count == -1) {
                throw new KryoBufferUnderflowException("Buffer underflow.");
            }
            remaining += count;
            if (remaining >= required) {
                limit += count;
                return remaining;
            }
        }

        // Was not enough, compact and try again.
        System.arraycopy(buffer, position, buffer, 0, remaining);
        total += position;
        position = 0;

        do {
            // Logical change 2 (from Kryo Input.require): "capacity - remaining" -> "required -
            // remaining"
            count = fill(buffer, remaining, required - remaining);
            if (count == -1) {

View on GitHub (pinned to 2f3c205e92)

Solutions

1. Verify the byte stream is complete and was produced by the matching serializer
2. Check length prefixes / record framing written upstream for off-by-one or truncation bugs
3. If restoring state, confirm the checkpoint file is fully persisted and not partially uploaded

try {
    T v = serializer.deserialize(inputView);
} catch (KryoException e) {
    if (e.getCause() instanceof EOFException || e.getMessage().contains("underflow")) {
        // truncated stream: fail the partition/read, do not retry blindly
        throw new IOException("Truncated serialized stream", e);
    }
    throw e;
}

When it happens

Trigger: Reading a Kryo-serialized record through NoFetchingInput when the DataInputView/source has fewer bytes remaining than the value being read needs - e.g. reading past the end of a serialized record boundary, or a truncated stream.

Common situations: Truncated checkpoint/snapshot files; length-prefixed records where the length is wrong (corruption or writer bug); deserializing a stream with a different format than expected; network stream cut mid-record.

Related errors

• Buffer too small: capacity: {}, required: {}
• No more bytes left.
• Reached the end of the collection. This could be caused by i
• Failed to serialize value '{value}'
• Cannot register null class or serializer.

AI-assisted analysis of apache/flink@2f3c205e92 (2026-08-14). Data as JSON: /api/errors/a3c938ef29cce371. Report an issue: GitHub.