apache/kafka · error · IllegalStateException

MockConsumer didn't have end offset specified, but tried to

Error message

MockConsumer didn't have end offset specified, but tried to seek to end

What it means

MockConsumer is a test double for KafkaConsumer that simulates offset resets without a broker. When the subscription's AutoOffsetResetStrategy is LATEST, MockConsumer looks up the partition's end offset from its internal endOffsets map (seeded by the test). If no end offset has been recorded for that TopicPartition, it throws this IllegalStateException because it cannot decide where to seek.

Source

Thrown at clients/src/main/java/org/apache/kafka/clients/consumer/MockConsumer.java:746

        } else if (!committed.containsKey(tp)) {
            subscriptions.requestOffsetReset(tp);
            resetOffsetPosition(tp);
        } else {
            subscriptions.seek(tp, committed.get(tp).offset());
        }
    }

    private void resetOffsetPosition(TopicPartition tp) {
        AutoOffsetResetStrategy strategy = subscriptions.resetStrategy(tp);
        Long offset;
        if (strategy == AutoOffsetResetStrategy.EARLIEST) {
            offset = beginningOffsets.get(tp);
            if (offset == null)
                throw new IllegalStateException("MockConsumer didn't have beginning offset specified, but tried to seek to beginning");
        } else if (strategy == AutoOffsetResetStrategy.LATEST) {
            offset = endOffsets.get(tp);
            if (offset == null)
                throw new IllegalStateException("MockConsumer didn't have end offset specified, but tried to seek to end");
        } else if (strategy.type() == AutoOffsetResetStrategy.StrategyType.BY_DURATION) {
            offset = durationResetOffsets.get(tp);
            if (offset == null)
                throw new IllegalStateException("MockConsumer didn't have duration offset specified, but tried to seek to timestamp");
        } else {
            throw new NoOffsetForPartitionException(tp);
        }
        seek(tp, offset);
    }

    @Override
    public List<PartitionInfo> partitionsFor(String topic, Duration timeout) {
        return partitionsFor(topic);
    }

    @Override
    public Map<String, List<PartitionInfo>> listTopics(Duration timeout) {
        return listTopics();

View on GitHub (pinned to 996fb4585a)

Solutions

1. Before poll(), call mockConsumer.updateEndOffsets(Map.of(tp, <endOffsetLong>)) for every partition the consumer will be assigned.
2. If you only need earliest behavior, construct the MockConsumer with OffsetResetStrategy.EARLIEST and call updateBeginningOffsets(...) instead.
3. If you do not want a reset to happen at all, seed a committed offset via commitSync(...) or call seek(tp, offset) explicitly before poll so resetOffsetPosition is never reached.
4. After any rebalance/subscription change in the test, re-seed end offsets for the newly assigned partitions.

Example fix

// before
MockConsumer<byte[],byte[]> c = new MockConsumer<>(OffsetResetStrategy.LATEST);
c.subscribe(Collections.singleton("t"));
c.poll(Duration.ofMillis(0)); // -> IllegalStateException

// after
MockConsumer<byte[],byte[]> c = new MockConsumer<>(OffsetResetStrategy.LATEST);
c.updateEndOffsets(Map.of(new TopicPartition("t",0), 10L));
c.subscribe(Collections.singleton("t"));
c.poll(Duration.ofMillis(0));

// Before poll(), ensure every assigned partition has an end offset seeded.
Set<TopicPartition> assigned = mockConsumer.assignment();
Map<TopicPartition, Long> missing = assigned.stream()
    .filter(tp -> !endOffsetsSeeded.contains(tp))
    .collect(Collectors.toMap(tp -> tp, tp -> 0L));
if (!missing.isEmpty()) {
    mockConsumer.updateEndOffsets(missing);
}

null

null

When it happens

Trigger: Calling poll()/position() on a MockConsumer whose subscription uses OffsetResetStrategy.LATEST (the default for new MockConsumer(OffsetResetStrategy.LATEST)), for a TopicPartition that was never seeded via updateEndOffsets(...), and for which no committed offset exists, so the consumer falls through to resetOffsetPosition(tp) and strategy == LATEST.

Common situations: Writing KafkaConsumer unit tests and forgetting to call updateEndOffsets(Map.of(tp, 0L)) before the first poll; switching the test from EARLIEST to LATEST without re-seeding offsets; or assigning a new partition mid-test (rebalance) whose end offset was never provided.

Related errors

• MockConsumer didn't have duration offset specified, but trie
• clientInstanceId not set
• Cannot add records for a topics that is not subscribed by th
• This consumer has already been closed.
• Invalid negative offset

AI-assisted analysis of apache/kafka@996fb4585a (2026-08-11). Data as JSON: /api/errors/d34130bd9a741391. Report an issue: GitHub.