apache/kafka · error · IllegalArgumentException

Invalid leader epoch {leaderEpoch} (must be non-negative)

Error message

Invalid leader epoch {leaderEpoch} (must be non-negative)

What it means

IllegalArgumentException thrown by Metadata.updateLastSeenEpochIfNewer when the supplied leaderEpoch is negative. Leader epochs are monotonic non-negative counters used by the client to detect stale metadata and partition leadership changes; a negative value indicates corrupt or fabricated protocol data. The guard rejects it before it can poison the lastSeenLeaderEpochs map.

Source

Thrown at clients/src/main/java/org/apache/kafka/clients/Metadata.java:235

        this.needPartialUpdate = true;
        this.equivalentResponseCount = 0;
        this.requestVersion++;
        return this.updateVersion;
    }

    /**
     * Request an update for the partition metadata if and only if we have seen a newer leader epoch. This is called by the client
     * any time it handles a response from the broker that includes leader epoch, except for update via Metadata RPC which
     * follows a different code path ({@link #update}).
     *
     * @param topicPartition The partition for which to update the last seen leader epoch.
     * @param leaderEpoch    The leader epoch received from the broker.
     * @return {@code true} if we updated the last seen epoch, {@code false} otherwise.
     */
    public synchronized boolean updateLastSeenEpochIfNewer(TopicPartition topicPartition, int leaderEpoch) {
        Objects.requireNonNull(topicPartition, "TopicPartition cannot be null");
        if (leaderEpoch < 0)
            throw new IllegalArgumentException("Invalid leader epoch " + leaderEpoch + " (must be non-negative)");

        Integer oldEpoch = lastSeenLeaderEpochs.get(topicPartition);
        log.trace("Determining if we should replace existing epoch {} with new epoch {} for partition {}", oldEpoch, leaderEpoch, topicPartition);

        final boolean updated;
        if (oldEpoch == null) {
            log.debug("Not replacing null epoch with new epoch {} for partition {}", leaderEpoch, topicPartition);
            updated = false;
        } else if (leaderEpoch > oldEpoch) {
            log.debug("Updating last seen epoch from {} to {} for partition {}", oldEpoch, leaderEpoch, topicPartition);
            lastSeenLeaderEpochs.put(topicPartition, leaderEpoch);
            updated = true;
        } else {
            log.debug("Not replacing existing epoch {} with new epoch {} for partition {}", oldEpoch, leaderEpoch, topicPartition);
            updated = false;
        }

        this.needFullUpdate = this.needFullUpdate || updated;

View on GitHub (pinned to c31c9215e1)

Solutions

Find the caller of updateLastSeenEpochIfNewer in the stack trace and confirm where the negative epoch originated.
If the value legitimately means 'no epoch', skip the call rather than passing -1 (guard on leaderEpoch >= 0 before invoking).
If the value comes from a broker response, capture the response and verify the broker is conformant — an unexpected negative epoch usually indicates a buggy/proxy response.
In tests, use the real broker epoch (>= 0) or omit the update.

Example fix

// before
metadata.updateLastSeenEpochIfNewer(tp, response.leaderEpoch()); // response.leaderEpoch() == -1
// after
if (response.leaderEpoch() >= 0) {
    metadata.updateLastSeenEpochIfNewer(tp, response.leaderEpoch());
}

Defensive patterns

Strategy: type-guard

Validate before calling

// Metadata.updateLastSeenEpochIfNewer rejects negative leaderEpoch.
// If your code derives an epoch from a committed offset or broker response,
// guard the value before passing it in. Use NO_PARTITION_LEADER_EPOCH (-1)
// or an empty Optional to mean 'unknown' rather than a negative number.
import org.apache.kafka.common.TopicPartition;
import java.util.Optional;

void updateEpoch(Metadata metadata, TopicPartition tp, int leaderEpoch) {
    if (leaderEpoch < 0) {
        // Unknown epoch — skip the update rather than throw.
        log.debug("Skipping epoch update for {}: no epoch known", tp);
        return;
    }
    metadata.updateLastSeenEpochIfNewer(tp, leaderEpoch);
}

Type guard

// Narrow an Optional<Integer> epoch down to a known-valid int before call.
static Optional<Integer> validLeaderEpoch(int raw) {
    return raw >= 0 ? Optional.of(raw) : Optional.empty();
}

// validLeaderEpoch(raw).ifPresent(e -> metadata.updateLastSeenEpochIfNewer(tp, e));

Try / catch

// IllegalArgumentException — programming bug, not a transient fault.
try {
    metadata.updateLastSeenEpochIfNewer(tp, leaderEpoch);
} catch (IllegalArgumentException e) {
    log.warn("Ignoring bad leader epoch {} for {}", leaderEpoch, tp, e);
    // do not retry with the same value; the value is wrong, not the call.
}

Prevention

Do not synthesise leader epochs yourself — they come from broker responses (MetadataResponse, OffsetForLeaderEpochResponse, FetchResponse). If you have no epoch, pass no epoch (absent Optional), not -1.
Treat any epoch < 0 as 'unknown' at every external boundary (offset stores, checkpoints, serializers) and normalise to Optional.empty().
Most application code never calls Metadata directly; if you find yourself doing so, prefer the higher-level Consumer/Producer/Admin APIs which manage epochs internally.
Unit-test your epoch-handling code with boundary inputs (-1, 0, Integer.MAX_VALUE).

When it happens

Trigger: A broker response, mocked test, or hand-crafted protocol message supplies a leaderEpoch < 0 (often -1) for a TopicPartition through any code path that calls updateLastSeenEpochIfNewer (consumer offsets, producer metadata, Fetch/Produce responses, Metadata cache updates).

Common situations: Test fixtures passing -1 to indicate 'unknown' instead of skipping the call; a custom broker or proxy emitting non-conformant responses; older client code upgraded against a newer broker whose epoch semantics changed; off-by-one in a serializer.

Related errors

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/d127d048ccc4c9c3.json. Report an issue: GitHub.