{"id":"c2fc2204be706f70","repo":"apache/kafka","slug":"invalid-magic-found-in-record","errorCode":null,"errorMessage":"Invalid magic found in record: {}","messagePattern":"Invalid magic found in record: (.+?)","errorType":"exception","errorClass":"CorruptRecordException","httpStatus":null,"severity":"critical","filePath":"clients/src/main/java/org/apache/kafka/common/record/internal/ByteBufferLogInputStream.java","lineNumber":84,"sourceCode":"    Integer nextBatchSize() throws CorruptRecordException {\n        int remaining = buffer.remaining();\n        if (remaining < LOG_OVERHEAD)\n            return null;\n        int recordSize = buffer.getInt(buffer.position() + SIZE_OFFSET);\n        // V0 has the smallest overhead, stricter checking is done later\n        if (recordSize < LegacyRecord.RECORD_OVERHEAD_V0)\n            throw new CorruptRecordException(String.format(\"Record size %d is less than the minimum record overhead (%d)\",\n                    recordSize, LegacyRecord.RECORD_OVERHEAD_V0));\n        if (recordSize > maxMessageSize)\n            throw new CorruptRecordException(String.format(\"Record size %d exceeds the largest allowable message size (%d).\",\n                    recordSize, maxMessageSize));\n\n        if (remaining < HEADER_SIZE_UP_TO_MAGIC)\n            return null;\n\n        byte magic = buffer.get(buffer.position() + MAGIC_OFFSET);\n        if (magic < 0 || magic > RecordBatch.CURRENT_MAGIC_VALUE)\n            throw new CorruptRecordException(\"Invalid magic found in record: \" + magic);\n\n        return recordSize + LOG_OVERHEAD;\n    }\n}\n","sourceCodeStart":66,"sourceCodeEnd":89,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/common/record/internal/ByteBufferLogInputStream.java#L66-L89","documentation":"Thrown by ByteBufferLogInputStream.nextBatchSize after reading the magic byte (buffer position + MAGIC_OFFSET) and finding a value that is negative or greater than RecordBatch.CURRENT_MAGIC_VALUE. The magic identifies the record-batch format version; only 0, 1, and 2 are defined. Anything else means the header is garbage, so the stream raises CorruptRecordException rather than guessing a format.","triggerScenarios":"nextBatchSize reads magic < 0 || magic > RecordBatch.CURRENT_MAGIC_VALUE (currently 2). Encountered by any code scanning a byte buffer as log batches — fetch handlers, replication, log recovery, kafka-dump-log.","commonSituations":"Truncated or partially written segment (magic byte not yet flushed); disk/filesystem corruption bit-flipping the magic; reading a non-Kafka file as a log; tail of a segment that was pre-allocated and not fully populated but read past its valid data.","solutions":["Confirm the segment is a real Kafka .log file and that you are not reading past the last valid batch (check the segment's recovered offset / truncated length).","Inspect with kafka-dump-log --files <segment.log> to locate the bad batch.","If genuinely corrupt, restore the segment from another replica or delete it so the broker truncates to the last valid offset."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// Magic byte is read from a potentially corrupt stream; pre-validation means\n// parsing it yourself first:\nbyte magic = buffer.get(buffer.position() + MAGIC_OFFSET);\nif (magic < 0 || magic > RecordBatch.CURRENT_MAGIC_VALUE) {\n    // unsupported/corrupt magic; stop reading this batch\n}","typeGuard":null,"tryCatchPattern":"try {\n    batch = stream.nextBatch();\n} catch (org.apache.kafka.common.errors.CorruptRecordException e) {\n    // unknown magic value; likely cross-version corruption or bit-rot.\n    // Skip and continue, or surface a hard failure depending on durability policy.\n}","preventionTips":["Only supported magic values are 0, 1, 2; anything else indicates corruption or a truncated/garbled segment.","Avoid mixing record formats in a single log segment across uncoordinated writes.","When upgrading message.format.version, do a controlled roll rather than interleaving versions."],"tags":["corrupt-record","magic-version","log-segment","storage"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}