{"id":"cde212d06456a353","repo":"apache/kafka","slug":"position-length-should-not-be-greater-than-buffer","errorCode":null,"errorMessage":"position+length should not be greater than buffer.limit(), position: \" + position + \", length: \" + length + \", buffer.limit(): \" + buffer.limit()","messagePattern":"position\\+length should not be greater than buffer\\.limit\\(\\), position: \" \\+ position \\+ \", length: \" \\+ length \\+ \", buffer\\.limit\\(\\): \" \\+ buffer\\.limit\\(\\)","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/common/record/internal/MemoryRecords.java","lineNumber":72,"sourceCode":"    private final Iterable<MutableRecordBatch> batches = this::batchIterator;\n\n    private int validBytes = -1;\n\n    // Construct a writable memory records\n    private MemoryRecords(ByteBuffer buffer) {\n        Objects.requireNonNull(buffer, \"buffer should not be null\");\n        this.buffer = buffer;\n    }\n\n    @Override\n    public int sizeInBytes() {\n        return buffer.limit();\n    }\n\n    @Override\n    public int writeTo(TransferableChannel channel, int position, int length) throws IOException {\n        if (((long) position) + length > buffer.limit())\n            throw new IllegalArgumentException(\"position+length should not be greater than buffer.limit(), position: \"\n                    + position + \", length: \" + length + \", buffer.limit(): \" + buffer.limit());\n\n        return Utils.tryWriteTo(channel, position, length, buffer);\n    }\n\n    /**\n     * Write all records to the given channel (including partial records).\n     * @param channel The channel to write to\n     * @return The number of bytes written\n     * @throws IOException For any IO errors writing to the channel\n     */\n    public int writeFullyTo(GatheringByteChannel channel) throws IOException {\n        buffer.mark();\n        int written = 0;\n        while (written < sizeInBytes())\n            written += channel.write(buffer);\n        buffer.reset();\n        return written;","sourceCodeStart":54,"sourceCodeEnd":90,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/common/record/internal/MemoryRecords.java#L54-L90","documentation":"Thrown by MemoryRecords.writeTo(channel, position, length) when the requested [position, position+length) byte range exceeds buffer.limit(). It guards the underlying transfer so callers cannot read past the end of the backing ByteBuffer. The message prints all three values (position, length, buffer.limit()) so the arithmetic gap is visible.","triggerScenarios":"Calling MemoryRecords.writeTo(...) with a (position,length) pair whose sum exceeds the sizeInBytes()/buffer.limit(). Typically reached during fetch/send paths, replica transfer, or any code that slices a MemoryRecords buffer and computes length from a stale or off-by-one size (e.g. using buffer.capacity() instead of limit(), or summing partial batch sizes).","commonSituations":"Off-by-one length after trimming trailing partial batch (length should be validBytes not sizeInBytes), buffer compaction that moved limit but caller still holds old size, position computed from a different buffer's index, or integer overflow when casting accumulated offsets.","solutions":["Use buffer.limit() (or MemoryRecords.sizeInBytes()/validBytes()) as the authoritative bound when computing length.","For partial trailing-batch trims, pass validBytes() rather than sizeInBytes().","Recompute position and length from the current buffer state immediately before the call rather than caching.","If summing batch sizes, guard against integer overflow and clamp to (limit - position)."],"exampleFix":"// before\nint len = buffer.capacity();\nrecords.writeTo(channel, position, len);\n// after\nint len = Math.min(requestedLen, buffer.limit() - position);\nrecords.writeTo(channel, position, len);","handlingStrategy":"validation","validationCode":"// MemoryRecords.writeTo requires position+length <= buffer.limit().\nint limit = records.sizeInBytes();\nif (position < 0 || length < 0 || (long) position + length > limit) {\n    throw new IllegalArgumentException(\n        \"Refusing writeTo: position=\" + position + \", length=\" + length + \", limit=\" + limit);\n}","typeGuard":"// Confirm the requested slice is within the records buffer before writing to a channel.\nstatic boolean withinBufferBounds(MemoryRecords records, int position, int length) {\n    int limit = records.sizeInBytes();\n    return position >= 0 && length >= 0 && (long) position + length <= limit;\n}","tryCatchPattern":"try {\n    int written = records.writeTo(channel, position, length);\n} catch (IllegalArgumentException e) {\n    if (e.getMessage().startsWith(\"position+length should not be greater than buffer.limit()\")) {\n        // Clamp length to remaining bytes and retry once.\n        length = records.sizeInBytes() - position;\n        written = records.writeTo(channel, position, length);\n    } else {\n        throw e;\n    }\n}","preventionTips":["Always compute length from records.sizeInBytes() - position rather than a cached/external total.","After any buffer flip/compact/slice operation, re-derive position and limit; never assume they survive across calls.","Treat position+length overflow as a bug in the caller's bookkeeping; log it and fix the source rather than silently clamping in production paths."],"tags":["kafka","records","buffer","network-transfer","bounds-check"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}