{"id":"9d2c5632c310e1b2","repo":"apache/kafka","slug":"found-invalid-record-count-in-magic-v-batch","errorCode":null,"errorMessage":"Found invalid record count {} in magic v{} batch","messagePattern":"Found invalid record count (.+?) in magic v(.+?) batch","errorType":"exception","errorClass":"InvalidRecordException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/common/record/internal/DefaultRecordBatch.java","lineNumber":586,"sourceCode":"    }\n\n    // visible for testing\n    abstract class RecordIterator implements CloseableIterator<Record> {\n        private final Long logAppendTime;\n        private final long baseOffset;\n        private final long baseTimestamp;\n        private final int baseSequence;\n        private final int numRecords;\n        private int readRecords = 0;\n\n        RecordIterator() {\n            this.logAppendTime = timestampType() == TimestampType.LOG_APPEND_TIME ? maxTimestamp() : null;\n            this.baseOffset = baseOffset();\n            this.baseTimestamp = baseTimestamp();\n            this.baseSequence = baseSequence();\n            int numRecords = count();\n            if (numRecords < 0)\n                throw new InvalidRecordException(\"Found invalid record count \" + numRecords + \" in magic v\" +\n                        magic() + \" batch\");\n            this.numRecords = numRecords;\n        }\n\n        @Override\n        public boolean hasNext() {\n            return readRecords < numRecords;\n        }\n\n        @Override\n        public Record next() {\n            if (readRecords >= numRecords)\n                throw new NoSuchElementException();\n\n            readRecords++;\n            Record rec = readNext(baseOffset, baseTimestamp, baseSequence, logAppendTime);\n            if (readRecords == numRecords) {\n                // Validate that the actual size of the batch is equal to declared size","sourceCodeStart":568,"sourceCodeEnd":604,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/common/record/internal/DefaultRecordBatch.java#L568-L604","documentation":"Thrown by DefaultRecordBatch.RecordIterator's constructor when the batch's declared record count (read via count() at the COUNT_OFFSET field) is negative. A negative count is structurally invalid — the count field encodes a non-negative varint/int of records — so this indicates either a corrupt batch or a buffer that was not actually a v2 batch. InvalidRecordException during iteration.","triggerScenarios":"Produced when iterator()/streamingIterator() is called on a DefaultRecordBatch whose COUNT_OFFSET int decodes to a negative value — e.g. reading a buffer whose header was zeroed/garbage, a partial write that left the count field unset, or pointing a DefaultRecordBatch at a ByteBuffer that is not a batch at all. The check at DefaultRecordBatch.java:585-587 precedes any record decoding.","commonSituations":"Log recovery scanning a segment with a torn or zeroed trailing header; consumer fetching from a replica that served a partially-written batch; custom tooling that wraps arbitrary ByteBuffers in DefaultRecordBatch without prior ensureValid; downgrade/upgrade where the count field semantics were misread.","solutions":["Call DefaultRecordBatch.ensureValid() before iterating; it catches size/CRC corruption earlier and isolates the bad batch.","Verify the buffer is actually a v2 batch (magic == 2 at MAGIC_OFFSET) before constructing DefaultRecordBatch.","If the segment is corrupt at the tail, recover via the broker's log-recovery truncation or remove the affected segment file.","In custom readers, do not advance past a batch whose length/count fields are inconsistent — re-fetch or skip rather than iterate."],"exampleFix":"// before: iterating an unvalidated, possibly-non-batch buffer\nDefaultRecordBatch b = new DefaultRecordBatch(buf);\nfor (Record r : b) { ... }\n\n// after: validate before iteration\nDefaultRecordBatch b = new DefaultRecordBatch(buf);\nif (b.magic() != RecordBatch.CURRENT_MAGIC_VALUE) throw new IOException(\"not a v2 batch\");\nb.ensureValid();\nfor (Record r : b) { ... }","handlingStrategy":"try-catch","validationCode":"// Record count is read out of the batch header on the READ path; user code does not pass it.\n// The only prevention on the produce side is to never forge batches with negative counts —\n// always use MemoryRecordsBuilder, which computes count correctly.","typeGuard":null,"tryCatchPattern":"// Invalid record count surfaces when iterating; skip and continue:\ntry {\n    batch.iterator().forEachRemaining(this::process);\n} catch (InvalidRecordException e) { // negative record count\n    log.error(\"Batch at {} @ {} has invalid record count, skipping\", partition, offset, e);\n    consumer.seek(partition, offset + 1);\n}","preventionTips":["Negative record count means the batch header itself is corrupt — same root causes as CRC errors (disk rot, truncation, bad producer).","Do not reuse a MemoryRecordsBuilder after reset without clearing the records count; double counting can leak invalid counts downstream.","If you see this consistently from one partition, broker log recovery is warranted — isolate the segment and run kafka-storage tooling to inspect it."],"tags":["kafka","record-format","consumer","broker","corruption","validation"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}