{"record":{"id":"2ba9796acc1e8b37","repo":"apache/flink","slug":"buffer-too-small-capacity-required","errorCode":null,"errorMessage":"Buffer too small: capacity: {}, required: {}","messagePattern":"Buffer too small: capacity: (.+?), required: (.+?)","errorType":"exception","errorClass":"KryoException","httpStatus":null,"severity":"error","filePath":"flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime/NoFetchingInput.java","lineNumber":73,"sourceCode":"     * then it will load exactly the difference between required and currently available number of\n     * bytes. Thus, it will only load the data which is required and never prefetch data.\n     *\n     * @param required the number of bytes being available in the buffer\n     * @return The number of bytes remaining in the buffer, which will be at least <code>required\n     *     </code> bytes.\n     * @throws KryoException\n     */\n    @Override\n    protected int require(int required) throws KryoException {\n        // The main change between this and Kryo 5 Input.require is this will never read more bytes\n        // than required.\n        // There are also formatting changes to be compliant with the Flink project styling rules.\n        int remaining = limit - position;\n        if (remaining >= required) {\n            return remaining;\n        }\n        if (required > capacity) {\n            throw new KryoException(\n                    \"Buffer too small: capacity: \" + capacity + \", required: \" + required);\n        }\n\n        int count;\n        // Try to fill the buffer.\n        if (remaining > 0) {\n            // Logical change 1 (from Kryo Input.require): \"capacity - limit\" -> \"required - limit\"\n            count = fill(buffer, limit, required - limit);\n            if (count == -1) {\n                throw new KryoBufferUnderflowException(\"Buffer underflow.\");\n            }\n            remaining += count;\n            if (remaining >= required) {\n                limit += count;\n                return remaining;\n            }\n        }\n","sourceCodeStart":55,"sourceCodeEnd":91,"githubUrl":"https://github.com/apache/flink/blob/2f3c205e9266cb30240eb7f4fdab15cad629a70f/flink-core/src/main/java/org/apache/flink/api/java/typeutils/runtime/NoFetchingInput.java#L55-L91","documentation":"NoFetchingInput.require(required) throws this KryoException when a single read needs more bytes than the buffer's total capacity. The adapter is constructed with capacity 8, so any Kryo primitive read requiring more than 8 contiguous bytes (e.g. a 9+ byte variable-length long, or a length-prefixed chunk larger than the buffer) cannot be satisfied.","triggerScenarios":"Deserializing through KryoDeserializer with NoFetchingInput when the data stream contains a varlong of 9 bytes (negative longs encode to 9-10 bytes in Kryo) or any read whose 'required' size exceeds 8; typically indicates a corrupted or mismatched stream rather than a legitimately oversized primitive.","commonSituations":"Stream corruption from partial writes or offset misalignment; data serialized by a different Kryo version/config (different varint encoding) than the reader; reading Kryo bytes not produced through the matching Flink serializer.","solutions":["Verify the data was written and read with the same serializer/Kryo configuration (registration ids, class-name vs registration mode)","Check for stream corruption: truncated checkpoints, shared mutable buffers, or offset bugs in the surrounding operator","Report upstream if the payload legitimately exceeds the fixed capacity - NoFetchingInput may need a larger buffer for such codecs"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    T value = kryoDeserializer.deserialize(input);\n} catch (KryoException e) {\n    if (e.getMessage().startsWith(\"Buffer too small\")) {\n        // required bytes exceeded NoFetchingInput capacity -> stream format mismatch or corruption\n        throw new IllegalStateException(\"Serialized stream incompatible with NoFetchingInput: \" + e.getMessage(), e);\n    }\n    throw e;\n}","preventionTips":["Write and read Kryo data with the same serializer configuration and Kryo version","Avoid encodings whose single-value reads exceed 8 bytes when passing through NoFetchingInput","Add integrity checks (lengths/checksums) around serialized payloads"],"tags":["kryo","serialization","buffer","stream-corruption"],"backgroundTag":null,"analyzedSha":"2f3c205e9266cb30240eb7f4fdab15cad629a70f","analyzedAt":"2026-08-14T08:48:24.518Z","schemaVersion":2},"datasetVersion":"2026-08-14T10:17:34.591Z"}