{"id":"c3169dd35c62530c","repo":"apache/kafka","slug":"record-size-d-is-less-than-the-minimum-record-ove","errorCode":null,"errorMessage":"Record size %d is less than the minimum record overhead (%d)","messagePattern":"Record size (.+?) is less than the minimum record overhead \\((.+?)\\)","errorType":"exception","errorClass":"CorruptRecordException","httpStatus":null,"severity":"critical","filePath":"clients/src/main/java/org/apache/kafka/common/record/internal/ByteBufferLogInputStream.java","lineNumber":73,"sourceCode":"            return new DefaultRecordBatch(batchSlice);\n        else\n            return new AbstractLegacyRecordBatch.ByteBufferLegacyRecordBatch(batchSlice);\n    }\n\n    /**\n     * Validates the header of the next batch and returns batch size.\n     * @return next batch size including LOG_OVERHEAD if buffer contains header up to\n     *         magic byte, null otherwise\n     * @throws CorruptRecordException if record size or magic is invalid\n     */\n    Integer nextBatchSize() throws CorruptRecordException {\n        int remaining = buffer.remaining();\n        if (remaining < LOG_OVERHEAD)\n            return null;\n        int recordSize = buffer.getInt(buffer.position() + SIZE_OFFSET);\n        // V0 has the smallest overhead, stricter checking is done later\n        if (recordSize < LegacyRecord.RECORD_OVERHEAD_V0)\n            throw new CorruptRecordException(String.format(\"Record size %d is less than the minimum record overhead (%d)\",\n                    recordSize, LegacyRecord.RECORD_OVERHEAD_V0));\n        if (recordSize > maxMessageSize)\n            throw new CorruptRecordException(String.format(\"Record size %d exceeds the largest allowable message size (%d).\",\n                    recordSize, maxMessageSize));\n\n        if (remaining < HEADER_SIZE_UP_TO_MAGIC)\n            return null;\n\n        byte magic = buffer.get(buffer.position() + MAGIC_OFFSET);\n        if (magic < 0 || magic > RecordBatch.CURRENT_MAGIC_VALUE)\n            throw new CorruptRecordException(\"Invalid magic found in record: \" + magic);\n\n        return recordSize + LOG_OVERHEAD;\n    }\n}\n","sourceCodeStart":55,"sourceCodeEnd":89,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/common/record/internal/ByteBufferLogInputStream.java#L55-L89","documentation":"Thrown by ByteBufferLogInputStream.nextBatchSize when the size read from the batch header (buffer position + SIZE_OFFSET) is smaller than LegacyRecord.RECORD_OVERHEAD_V0 — the absolute minimum byte overhead a v0 record can occupy. A size below this floor is structurally impossible for any valid record, so the stream treats the segment as corrupt and raises CorruptRecordException. V0 has the smallest overhead, so anything smaller is unambiguously garbage.","triggerScenarios":"Reading a log segment via ByteBufferLogInputStream where the 4-byte size field decodes to a value below LegacyRecord.RECORD_OVERHEAD_V0. Produced by FileLogInputStream/byte-buffer scanning during fetch, replication, or log recovery.","commonSituations":"On-disk log corruption (partial write, fsync failure, disk fault, unclean shutdown); truncated segment tail; reading a file that is not actually a Kafka log; mismatched segment offset/index files after a manual copy.","solutions":["Delete the corrupt segment (.log + .index/.timeindex) and let Kafka recover, or restore from a known-good replica.","Run kafka-dump-log --files <segment.log> --print-data-log to inspect the offending batch and confirm the size field.","Ensure the broker shut down cleanly and the disk/fs is healthy; check dmesg / filesystem logs for underlying I/O errors."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// This is raised on read from a (corrupt) byte stream; you cannot pre-validate\n// without parsing. If you control the buffer, sanity-check the size field:\nint recordSize = buffer.getInt(buffer.position() + SIZE_OFFSET);\nif (recordSize < LegacyRecord.RECORD_OVERHEAD_V0) {\n    // corrupt/truncated log segment; stop iterating\n}","typeGuard":null,"tryCatchPattern":"try {\n    batch = stream.nextBatch();\n} catch (org.apache.kafka.common.errors.CorruptRecordException e) {\n    // log segment is corrupt; skip batch, advance position, or halt recovery\n    log.warn(\"Corrupt record (size below overhead), skipping\", e);\n}","preventionTips":["Treat CorruptRecordException as a symptom of disk/replication corruption, not caller error.","Validate checksums / use replication to detect bit-rot before reading.","When iterating a log, be prepared to skip past corrupt batches rather than aborting the whole scan."],"tags":["corrupt-record","log-segment","record-format","storage"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}