apache/kafka · error · IllegalArgumentException

Invalid negative offset

Error message

Invalid negative offset

What it means

OffsetAndMetadata is the value object stored for each partition when committing offsets (and serialized in OffsetCommit requests). The constructor enforces offset >= 0 because negative offsets are not representable on the broker and would corrupt committed state. The check throws IllegalArgumentException before any field is assigned.

Solutions

1. Validate offset >= 0 before constructing OffsetAndMetadata; skip the commit for that partition if offset is unknown.
2. Replace sentinel -1 semantics with Optional<Long> / null and only construct OffsetAndMetadata when present.
3. If committing KafkaConsumer.position() directly, ensure the consumer has a position (position() returns >= 0 once assigned and polled).

Example fix

// before
long off = position > 0 ? position - 1 : -1;
consumer.commitSync(Map.of(tp, new OffsetAndMetadata(off, ""))); // -> IllegalArgumentException when position==0

// after
if (position < 0) {
    // skip commit, no position yet
} else {
    consumer.commitSync(Map.of(tp, new OffsetAndMetadata(position, "")));
}

if (offset < 0) throw new IllegalStateException("no offset to commit for " + tp);
return new OffsetAndMetadata(offset, leaderEpoch, metadata);

static boolean isCommittableOffset(long offset) { return offset >= 0; }

try { consumer.commitSync(Map.of(tp, new OffsetAndMetadata(off, ""))); }
catch (IllegalArgumentException e) { /* skip this partition's commit */ }

When it happens

Trigger: new OffsetAndMetadata(-1, ...), or any construction path where the offset arithmetic underflows (e.g., position-1 when position==0, or a sentinel value of -1 used to mean 'no offset').

Common situations: Off-by-one in 'last consumed offset + 1' logic; passing a 'not found' sentinel (-1) from a custom store into commitSync; replaying offsets read from an external system that uses -1 for 'unknown'.

Related errors

• Invalid negative offset
• Invalid negative timestamp
• The configured group.id should not be an empty string or…
• cannot be set when using a share group.
• Consumer is not subscribed to any topics or assigned any…

AI-assisted analysis of apache/kafka@996fb4585a (2026-08-11). Data as JSON: /api/errors/c597cf296973bd6c. Report an issue: GitHub.

Appendix: source

Thrown at clients/src/main/java/org/apache/kafka/clients/consumer/OffsetAndMetadata.java:52

    private final long offset;
    private final String metadata;

    // We use null to represent the absence of a leader epoch to simplify serialization.
    // I.e., older serializations of this class which do not have this field will automatically
    // initialize its value to null.
    private final Integer leaderEpoch;

    /**
     * Construct a new OffsetAndMetadata object for committing through {@link KafkaConsumer}.
     *
     * @param offset The offset to be committed
     * @param leaderEpoch Optional leader epoch of the last consumed record
     * @param metadata Non-null metadata
     */
    public OffsetAndMetadata(long offset, Optional<Integer> leaderEpoch, String metadata) {
        if (offset < 0)
            throw new IllegalArgumentException("Invalid negative offset");

        this.offset = offset;
        this.leaderEpoch = leaderEpoch.orElse(null);

        // The server converts null metadata to an empty string. So we store it as an empty string as well on the client
        // to be consistent.
        this.metadata = Objects.requireNonNullElse(metadata, OffsetFetchResponse.NO_METADATA);
    }

    /**
     * Construct a new OffsetAndMetadata object for committing through {@link KafkaConsumer}.
     * @param offset The offset to be committed
     * @param metadata Non-null metadata
     */
    public OffsetAndMetadata(long offset, String metadata) {
        this(offset, Optional.empty(), metadata);
    }

View on GitHub (pinned to 996fb4585a)