{"id":"9b46f0949c7e206d","repo":"apache/kafka","slug":"found-invalid-wrapper-offset-in-compressed-v1-mess","errorCode":null,"errorMessage":"Found invalid wrapper offset in compressed v1 message set, wrapper offset '{}' is less than the last inner message offset '{}' and it is not zero.","messagePattern":"Found invalid wrapper offset in compressed v1 message set, wrapper offset '(.+?)' is less than the last inner message offset '(.+?)' and it is not zero\\.","errorType":"exception","errorClass":"InvalidRecordException","httpStatus":null,"severity":"critical","filePath":"clients/src/main/java/org/apache/kafka/common/record/internal/AbstractLegacyRecordBatch.java","lineNumber":381,"sourceCode":"                                timestampFromWrapper,\n                                wrapperRecord.timestampType());\n                        innerEntry = new BasicLegacyRecordBatch(innerEntry.lastOffset(), recordWithTimestamp);\n                    }\n\n                    innerEntries.addLast(innerEntry);\n                }\n\n                if (innerEntries.isEmpty())\n                    throw new InvalidRecordException(\"Found invalid compressed record set with no inner records\");\n\n                if (wrapperMagic == RecordBatch.MAGIC_VALUE_V1) {\n                    if (lastOffsetFromWrapper == 0) {\n                        // The outer offset may be 0 if this is produce data from certain versions of librdkafka.\n                        this.absoluteBaseOffset = 0;\n                    } else {\n                        long lastInnerOffset = innerEntries.getLast().offset();\n                        if (lastOffsetFromWrapper < lastInnerOffset)\n                            throw new InvalidRecordException(\"Found invalid wrapper offset in compressed v1 message set, \" +\n                                    \"wrapper offset '\" + lastOffsetFromWrapper + \"' is less than the last inner message \" +\n                                    \"offset '\" + lastInnerOffset + \"' and it is not zero.\");\n                        this.absoluteBaseOffset = lastOffsetFromWrapper - lastInnerOffset;\n                    }\n                } else {\n                    this.absoluteBaseOffset = -1;\n                }\n            } catch (IOException e) {\n                throw new KafkaException(e);\n            } finally {\n                Utils.closeQuietly(stream, \"records iterator stream\");\n            }\n        }\n\n        @Override\n        protected Record makeNext() {\n            if (innerEntries.isEmpty())\n                return allDone();","sourceCodeStart":363,"sourceCodeEnd":399,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/common/record/internal/AbstractLegacyRecordBatch.java#L363-L399","documentation":"Thrown by DeepRecordsIterator (line 381) as an InvalidRecordException for magic v1 compressed message sets when the wrapper's lastOffset is non-zero but smaller than the last inner record's offset. v1 uses relative offsets: wrapper.lastOffset must equal absolute last offset, and the inner last offset is a relative value, so wrapper offset < inner last offset is impossible for well-formed data unless the wrapper offset was rewritten incorrectly.","triggerScenarios":"A compressed v1 batch whose wrapper offset was reassigned (e.g. by a buggy broker assignor or replication path) to a value smaller than the relative offset of its last inner record, without being reset to zero. Also triggered by tooling that rewrites offsets (kafka-reassign-partitions, mirror-maker, custom offset translators) incorrectly.","commonSituations":"MirrorMaker or other offset-rewriting pipelines that lower the wrapper offset without recomputing inner offsets. A broker bug in offset assignment during log compaction or replication of v1 compressed batches. Custom tooling that edits the offset field of a compressed v1 batch in place.","solutions":["Do not rewrite offsets on compressed v1 batches in custom tooling; let the broker assign offsets at produce time.","Migrate the topic to message.format.version 0.11+ (v2), which uses a base offset + count scheme that is far less prone to this class of bug.","If the bad data is historical, identify the segment with kafka-dump-log --deep-iteration, drop the corrupt range, and restore from a clean replica or backup.","Upgrade any offset-rewriting component (MirrorMaker, replication tools) to a version that correctly handles v1 compressed batches or that converts to v2."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":null,"typeGuard":null,"tryCatchPattern":"// For v1 compressed batches the wrapper offset must be >= the last inner offset\n// (or exactly 0). A violation means offsets are corrupt.\nimport org.apache.kafka.common.errors.InvalidRecordException;\n\ntry {\n    for (Record r : legacyBatch) { /* process */ }\n} catch (InvalidRecordException e) {\n    log.error(\"Legacy v1 compressed batch has inconsistent wrapper/inner offsets; skipping\", e);\n}","preventionTips":["Offsets are assigned by the broker; never rewrite wrapper or inner offsets client-side on compressed legacy batches.","Avoid using non-Apache-Kafka producers that fabricate offsets for compressed v1 batches; the librdkafka offset-0 special case is the only tolerated exception.","When migrating offsets or repairing log segments, recompute the wrapper offset as last-inner-offset + (count-1) plus the absolute base, or use the broker's offset-assignment path.","Treat the batch as corrupt; skip and alert rather than retry."],"tags":["offset","compression","magic-v1","legacy-record","replication","validation"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}