{"id":"caddee42af2d1fde","repo":"apache/kafka","slug":"record-size-d-exceeds-the-largest-allowable-messa","errorCode":null,"errorMessage":"Record size %d exceeds the largest allowable message size (%d).","messagePattern":"Record size (.+?) exceeds the largest allowable message size \\((.+?)\\)\\.","errorType":"exception","errorClass":"CorruptRecordException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/common/record/internal/ByteBufferLogInputStream.java","lineNumber":76,"sourceCode":"    }\n\n    /**\n     * Validates the header of the next batch and returns batch size.\n     * @return next batch size including LOG_OVERHEAD if buffer contains header up to\n     *         magic byte, null otherwise\n     * @throws CorruptRecordException if record size or magic is invalid\n     */\n    Integer nextBatchSize() throws CorruptRecordException {\n        int remaining = buffer.remaining();\n        if (remaining < LOG_OVERHEAD)\n            return null;\n        int recordSize = buffer.getInt(buffer.position() + SIZE_OFFSET);\n        // V0 has the smallest overhead, stricter checking is done later\n        if (recordSize < LegacyRecord.RECORD_OVERHEAD_V0)\n            throw new CorruptRecordException(String.format(\"Record size %d is less than the minimum record overhead (%d)\",\n                    recordSize, LegacyRecord.RECORD_OVERHEAD_V0));\n        if (recordSize > maxMessageSize)\n            throw new CorruptRecordException(String.format(\"Record size %d exceeds the largest allowable message size (%d).\",\n                    recordSize, maxMessageSize));\n\n        if (remaining < HEADER_SIZE_UP_TO_MAGIC)\n            return null;\n\n        byte magic = buffer.get(buffer.position() + MAGIC_OFFSET);\n        if (magic < 0 || magic > RecordBatch.CURRENT_MAGIC_VALUE)\n            throw new CorruptRecordException(\"Invalid magic found in record: \" + magic);\n\n        return recordSize + LOG_OVERHEAD;\n    }\n}\n","sourceCodeStart":58,"sourceCodeEnd":89,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/common/record/internal/ByteBufferLogInputStream.java#L58-L89","documentation":"Thrown by ByteBufferLogInputStream.nextBatchSize when the record size decoded from the header exceeds the maxMessageSize limit the stream was constructed with. The cap is the broker/client configured message ceiling (message.max.bytes / max.message.bytes); a record claiming to be larger is either corrupt or indicates the producer and reader disagree on the configured limit. Treated as CorruptRecordException because a valid stream cannot contain a batch larger than the negotiated maximum.","triggerScenarios":"nextBatchSize reads a size value greater than the maxMessageSize passed into ByteBufferLogInputStream's constructor. Happens during fetch/replication/log-recovery when the producing side wrote a batch larger than the reading side's maxMessageSize, or when the size field is itself corrupt.","commonSituations":"Broker A has message.max.bytes=2MB and produces a large batch; broker B / consumer / replica is configured with a smaller max and rejects it. Also seen when a size field is bit-flipped/corrupt, inflating the value.","solutions":["Align max message size across producer, broker (message.max.bytes), topic (max.message.bytes), and consumer (max.partition.fetch.bytes / fetch.max.bytes), then restart the affected component.","Verify with kafka-dump-log whether the size field reflects a real large batch (config mismatch) or an implausible value (corruption).","If corrupt, delete/restore the offending segment as in the underflow case."],"exampleFix":"# before\nmessage.max.bytes=1000012\n\n# after (match across producer/broker/topic/consumer)\nmessage.max.bytes=8388608\nmax.message.bytes=8388608\nmax.request.size=8388608\nfetch.max.bytes=8388608","handlingStrategy":"try-catch","validationCode":"// The limit comes from maxMessageSize passed to ByteBufferLogInputStream.\n// Ensure it matches broker/message max (message.max.bytes / max.message.bytes):\nint maxMessageSize = Math.max(configuredMax, recordSize + LOG_OVERHEAD);\n// If you already know recordSize, you can short-circuit:\nif (recordSize > maxMessageSize) {\n    // oversized or corrupt; do not attempt to materialize\n}","typeGuard":null,"tryCatchPattern":"try {\n    batch = stream.nextBatch();\n} catch (org.apache.kafka.common.errors.CorruptRecordException e) {\n    // either a genuinely oversized message or corruption;\n    // check message.max.bytes on producer, broker, and consumer configs\n}","preventionTips":["Align max.message.bytes across producer, broker, and consumer to avoid truncation reads.","A size above maxMessageSize usually means the producer was misconfigured or the segment is corrupt; investigate the source.","Cap serialized record size client-side before send() to fail fast instead of on read."],"tags":["corrupt-record","message-size","configuration","log-segment"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}