{"id":"be5905219042b8d5","repo":"apache/kafka","slug":"found-invalid-number-of-record-headers","errorCode":null,"errorMessage":"Found invalid number of record headers {}","messagePattern":"Found invalid number of record headers (.+?)","errorType":"exception","errorClass":"InvalidRecordException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/common/record/internal/DefaultRecord.java","lineNumber":340,"sourceCode":"                timestamp = logAppendTime;\n\n            int offsetDelta = ByteUtils.readVarint(buffer);\n            long offset = baseOffset + offsetDelta;\n            int sequence = baseSequence >= 0 ?\n                    DefaultRecordBatch.incrementSequence(baseSequence, offsetDelta) :\n                    RecordBatch.NO_SEQUENCE;\n\n            // read key\n            int keySize = ByteUtils.readVarint(buffer);\n            ByteBuffer key = Utils.readBytes(buffer, keySize);\n\n            // read value\n            int valueSize = ByteUtils.readVarint(buffer);\n            ByteBuffer value = Utils.readBytes(buffer, valueSize);\n\n            int numHeaders = ByteUtils.readVarint(buffer);\n            if (numHeaders < 0)\n                throw new InvalidRecordException(\"Found invalid number of record headers \" + numHeaders);\n            if (numHeaders > buffer.remaining())\n                throw new InvalidRecordException(\"Found invalid number of record headers. \" + numHeaders + \" is larger than the remaining size of the buffer\");\n\n            final Header[] headers;\n            if (numHeaders == 0)\n                headers = Record.EMPTY_HEADERS;\n            else\n                headers = readHeaders(buffer, numHeaders);\n\n            // validate whether we have read all header bytes in the current record\n            if (buffer.position() - recordStart != sizeOfBodyInBytes)\n                throw new InvalidRecordException(\"Invalid record size: expected to read \" + sizeOfBodyInBytes +\n                        \" bytes in record payload, but instead read \" + (buffer.position() - recordStart));\n\n            int totalSizeInBytes = ByteUtils.sizeOfVarint(sizeOfBodyInBytes) + sizeOfBodyInBytes;\n            return new DefaultRecord(totalSizeInBytes, attributes, offset, timestamp, sequence, key, value, headers);\n        } catch (BufferUnderflowException | IllegalArgumentException e) {\n            throw new InvalidRecordException(\"Found invalid record structure\", e);","sourceCodeStart":322,"sourceCodeEnd":358,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/common/record/internal/DefaultRecord.java#L322-L358","documentation":"Thrown on the ByteBuffer deserialization path when the varint read for the per-record headers count is negative (numHeaders < 0). Magic-v2 records encode the header count as a varint and a negative value means the bytes are not a valid count; the library refuses to allocate a negative-sized header array.","triggerScenarios":"Raised at DefaultRecord.java:339-340 inside the private readFrom(ByteBuffer, ...) after ByteUtils.readVarint(buffer) at line 338 returns a negative number. Triggered by any corrupt or hand-crafted batch where the bytes that should encode numHeaders instead decode as -1 (the null-marker) or another negative varint.","commonSituations":"A producer wrote a record with a malformed header section (e.g. via a custom Serializer that bypassed DefaultRecord.writeTo), bit-flip on disk or in transit, partial overwrite of a log segment, or an attempt to consume a topic produced by a non-Apache-Kafka client with an off-spec record format.","solutions":["Use kafka-dump-log --files <segment> --print-data-log to locate the record with the bad header count and identify the producing client.","Ensure the producer actually uses the Apache Kafka serializer path (ProducerRecord + DefaultRecord.writeTo) rather than writing raw bytes; never hand-format the headers section.","If caused by disk corruption, verify checksums (DefaultRecordBatch validates CRC) and recover the partition from a healthy in-sync replica.","Add a producer-side integration test that round-trips records with the same header set the consumer reads, to catch off-spec serializers before deploy."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"// numHeaders is decoded internally; you can only pre-check that the varint\n// stream still has room. Full prevention requires re-reading the varint\n// yourself, which duplicates the parser. Rely on try-catch.\n// If you do decode numHeaders manually:\nif (numHeaders < 0) { /* malformed; skip record */ }","typeGuard":"// Guard a decoded header count before further parsing.\nprivate static boolean isValidHeaderCount(int numHeaders) {\n    return numHeaders >= 0;\n}","tryCatchPattern":"try {\n    DefaultRecord r = DefaultRecord.readFrom(buffer, baseOffset, baseTimestamp, baseSequence, logAppendTime);\n} catch (InvalidRecordException e) {\n    // negative numHeaders means the varint field was corrupted\n    LOG.warn(\"Malformed record (bad header count) near offset {}\", baseOffset, e);\n}","preventionTips":["Use only Kafka-native serializers (ByteArraySerializer, StringSerializer, etc.) so the varint header-count field is always well-formed.","Do not bit-edit produced record bytes; the header-count varint is positional and a one-byte shift produces a huge or negative value.","Validate record headers on the producer side (count, key length) before send to surface corruption early."],"tags":["kafka","record-format","deserialization","headers","invalid-record"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}