{"id":"f586961c70754365","repo":"apache/kafka","slug":"record-size-is-less-than-the-minimum-record-overhe","errorCode":null,"errorMessage":"Record size is less than the minimum record overhead (%d)","messagePattern":"Record size is less than the minimum record overhead \\((.+?)\\)","errorType":"exception","errorClass":"CorruptRecordException","httpStatus":null,"severity":"critical","filePath":"clients/src/main/java/org/apache/kafka/common/record/internal/AbstractLegacyRecordBatch.java","lineNumber":301,"sourceCode":"        private final int maxMessageSize;\n        private final ByteBuffer offsetAndSizeBuffer;\n\n        DataLogInputStream(InputStream stream, int maxMessageSize) {\n            this.stream = stream;\n            this.maxMessageSize = maxMessageSize;\n            this.offsetAndSizeBuffer = ByteBuffer.allocate(Records.LOG_OVERHEAD);\n        }\n\n        public AbstractLegacyRecordBatch nextBatch() throws IOException {\n            offsetAndSizeBuffer.clear();\n            Utils.readFully(stream, offsetAndSizeBuffer);\n            if (offsetAndSizeBuffer.hasRemaining())\n                return null;\n\n            long offset = offsetAndSizeBuffer.getLong(Records.OFFSET_OFFSET);\n            int size = offsetAndSizeBuffer.getInt(Records.SIZE_OFFSET);\n            if (size < LegacyRecord.RECORD_OVERHEAD_V0)\n                throw new CorruptRecordException(String.format(\"Record size is less than the minimum record overhead (%d)\", LegacyRecord.RECORD_OVERHEAD_V0));\n            if (size > maxMessageSize)\n                throw new CorruptRecordException(String.format(\"Record size exceeds the largest allowable message size (%d).\", maxMessageSize));\n\n            ByteBuffer batchBuffer = ByteBuffer.allocate(size);\n            Utils.readFully(stream, batchBuffer);\n            if (batchBuffer.hasRemaining())\n                return null;\n            batchBuffer.flip();\n\n            return new BasicLegacyRecordBatch(offset, new LegacyRecord(batchBuffer));\n        }\n    }\n\n    private static class DeepRecordsIterator extends AbstractIterator<Record> implements CloseableIterator<Record> {\n        private final ArrayDeque<AbstractLegacyRecordBatch> innerEntries;\n        private final long absoluteBaseOffset;\n        private final byte wrapperMagic;\n","sourceCodeStart":283,"sourceCodeEnd":319,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/common/record/internal/AbstractLegacyRecordBatch.java#L283-L319","documentation":"Thrown by DataLogInputStream.nextBatch() (line 301) as a CorruptRecordException when the size field read from a legacy (magic 0/1) message frame is smaller than LegacyRecord.RECORD_OVERHEAD_V0. The declared size is too small to even hold the fixed legacy record header, so the byte stream cannot contain a well-formed record. Kafka treats this as a corrupt-log condition that should cause the consumer to skip or fail the fetch depending on the broker's handling.","triggerScenarios":"Reading a corrupted or partially-written log segment through FileRecords / a LogInputStream, or feeding a truncated ByteBuffer into a legacy record iterator. Common when a broker crashed mid-flush, when on-disk data was truncated externally, or when an off-by-one frame parser feeds DataLogInputStream garbage.","commonSituations":"Broker crash or power loss during log append leaving a torn last record. Disk corruption or filesystem-level truncation of a .log segment. Custom producers that hand-craft message frames with a wrong size field. Mixing non-Kafka bytes into a file treated as a Kafka log.","solutions":["Run kafka-dump-log on the suspect segment to confirm corruption and locate the torn record.","Delete or truncate the corrupt segment (stop the broker, move the .log/.index/.timeindex aside, restart) so replication can rebuild it.","If reading from a ByteBuffer you constructed, verify the framing logic against Records.LOG_OVERHEAD + LegacyRecord.RECORD_OVERHEAD_V0 before calling nextBatch().","Ensure brokers shut down cleanly and use replication factor >= 2 so a healthy replica can replace the corrupt one."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"// CorruptRecordException is thrown while iterating a legacy (v0/v1) record stream.\n// You cannot pre-validate byte sizes without reading the stream, so wrap iteration.\nimport org.apache.kafka.common.errors.CorruptRecordException;\n\ntry {\n    for (RecordBatch batch : records.batches()) {\n        for (Record r : batch) { /* process */ }\n    }\n} catch (CorruptRecordException e) {\n    // record is shorter than RECORD_OVERHEAD_V0 => truncated/garbled segment.\n    // quarantine the segment, advance the consumer past it, or alert.\n    log.error(\"Corrupt legacy record (size < min overhead) in {}\", topicPartition, e);\n}","preventionTips":["These errors only occur with legacy message formats (magic v0/v1); use the modern v2 format (broker default since Kafka 0.11) to avoid this code path entirely.","Ensure brokers are not configured with message.format.version below your producer's format, which can trigger legacy parsing.","If you must read legacy segments, treat CorruptRecordException as a signal to advance the consumer offset or quarantine the segment rather than retrying the same bytes.","Run kafka-verify-offset / log sanity checks after unclean shutdowns or disk errors so truncated segments are detected before consumers hit them.","Do not hand-craft byte buffers for legacy records; always use the provided Record/RecordBatch builders."],"tags":["corruption","legacy-record","log-segment","consumer","magic-v0-v1"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}