{"id":"f28572ba0f742478","repo":"apache/kafka","slug":"invalid-offset-offsetandmetadata-offset","errorCode":null,"errorMessage":"Invalid offset: ${offsetAndMetadata.offset()}","messagePattern":"Invalid offset: (.+?)","errorType":"validation","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/CommitEvent.java","lineNumber":57,"sourceCode":"    protected final CompletableFuture<Void> offsetsReady = new CompletableFuture<>();\n\n    protected CommitEvent(final Type type, final Optional<Map<TopicPartition, OffsetAndMetadata>> offsets, final long deadlineMs) {\n        super(type, deadlineMs);\n        this.offsets = validate(offsets);\n    }\n\n    /**\n     * Validates the offsets are not negative and then returns the given offset map as\n     * {@link Collections#unmodifiableMap(Map) as unmodifiable}.\n     */\n    private static Optional<Map<TopicPartition, OffsetAndMetadata>> validate(final Optional<Map<TopicPartition, OffsetAndMetadata>> offsets) {\n        if (offsets.isEmpty()) {\n            return Optional.empty();\n        }\n\n        for (OffsetAndMetadata offsetAndMetadata : offsets.get().values()) {\n            if (offsetAndMetadata.offset() < 0) {\n                throw new IllegalArgumentException(\"Invalid offset: \" + offsetAndMetadata.offset());\n            }\n        }\n\n        return Optional.of(Collections.unmodifiableMap(offsets.get()));\n    }\n\n    public Optional<Map<TopicPartition, OffsetAndMetadata>> offsets() {\n        return offsets;\n    }\n\n    public CompletableFuture<Void> offsetsReady() {\n        return offsetsReady;\n    }\n\n    public void markOffsetsReady() {\n        offsetsReady.complete(null);\n    }\n","sourceCodeStart":39,"sourceCodeEnd":75,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/clients/consumer/internals/events/CommitEvent.java#L39-L75","documentation":"Thrown by CommitEvent.validate() when committing offsets and at least one OffsetAndMetadata in the supplied map has a negative offset(). Kafka offsets are non-negative sequence numbers within a partition; a negative value is never a legal commit target and almost always indicates a logic bug in the caller. The check runs in the CommitEvent constructor on the application thread, before the commit is ever enqueued to the network thread.","triggerScenarios":"consumer.commitSync(Map<TopicPartition,OffsetAndMetadata>) or commitAsync(...) with a hand-built map containing a negative offset; building OffsetAndMetadata from an arithmetic expression (e.g. position()-1) that underflows; using -1 as an 'unset' sentinel and passing it through to commit.","commonSituations":"Custom offset-tracking logic that subtracts from position()/offset and goes below zero on the first message; porting code that used -1 as a null marker; replay-from-offset tooling that loads offsets from a corrupt store; off-by-one in 'last committed + delta' calculations when the partition is empty.","solutions":["Validate/clamp offsets to >= 0 before constructing OffsetAndMetadata (Math.max(0, computed)).","Audit the source of the negative value — it is usually position()-N or end-begin underflow on an empty partition; guard those paths.","If you intended 'no offset', skip that partition from the commit map rather than sending a sentinel.","Add a unit test asserting every offset you commit is non-negative."],"exampleFix":"// before\nlong base = consumer.position(tp);\nlong toCommit = base - 1; // becomes -1 on first poll\nconsumer.commitSync(Map.of(tp, new OffsetAndMetadata(toCommit)));\n\n// after\nlong toCommit = Math.max(0, consumer.position(tp));\nconsumer.commitSync(Map.of(tp, new OffsetAndMetadata(toCommit)));","handlingStrategy":"validation","validationCode":"// Validate offsets are non-negative before handing them to commitSync /\n// commitAsync. CommitEvent.validate throws IllegalArgumentException on offset < 0.\nimport org.apache.kafka.clients.consumer.OffsetAndMetadata;\nimport org.apache.kafka.common.TopicPartition;\nimport java.util.Map;\n\nstatic void ensureValidOffsets(Map<TopicPartition, OffsetAndMetadata> offsets) {\n    for (Map.Entry<TopicPartition, OffsetAndMetadata> e : offsets.entrySet()) {\n        OffsetAndMetadata om = e.getValue();\n        if (om == null || om.offset() < 0) {\n            throw new IllegalArgumentException(\n                \"Refusing to commit invalid offset for \" + e.getKey()\n                + \": \" + (om == null ? \"null\" : om.offset()));\n        }\n    }\n}\n\n// usage:\nensureValidOffsets(offsets);\nconsumer.commitSync(offsets);","typeGuard":"// Predicate narrowing a raw offset map to a known-safe one (Java has no\n// structural type narrowing, so emulate with a validated wrapper).\nstatic boolean allOffsetsValid(Map<TopicPartition, OffsetAndMetadata> m) {\n    return m != null && m.values().stream()\n        .allMatch(om -> om != null && om.offset() >= 0);\n}\n// if (allOffsetsValid(offsets)) consumer.commitSync(offsets); else ...","tryCatchPattern":"try {\n    consumer.commitSync(offsets);\n} catch (IllegalArgumentException e) {\n    // offset < 0 slipped through; drop or reset the offending partition\n    log.warn(\"Rejected bad offset on commit; skipping: {}\", e.getMessage());\n}","preventionTips":["Always derive committed offsets from position() rather than computing them by hand, so they stay >= 0.","Centralize commit calls behind a helper that validates offsets >= 0 first.","Treat a negative offset as a data bug: log the TopicPartition and reset position before retrying.","When committing consumed offsets, clamp/validate immediately after computing them, not at the call site."],"tags":["consumer","offsets","commit","validation"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}