What causes "Missing position for fetchable partition {}" in apache/kafka?

Ensure every partition you assign manually is seek()'d (or has committed offsets) before the first poll().

apache/kafka · error · IllegalStateException

Missing position for fetchable partition {}

Error message

Missing position for fetchable partition {}

What it means

IllegalStateException("Missing position for fetchable partition X") thrown inside FetchCollector.fetchRecords when a partition is assigned and fetchable but subscriptions.position(tp) returns null. The consumer cannot determine the offset at which to consume returned records, indicating an internal state-machine inconsistency between assignment and position tracking rather than a normal application error.

Source

Thrown at clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchCollector.java:166

    private Fetch<K, V> fetchRecords(final CompletedFetch nextInLineFetch, int maxRecords) {
        final TopicPartition tp = nextInLineFetch.partition;

        if (!subscriptions.isAssigned(tp)) {
            // this can happen when a rebalance happened before fetched records are returned to the consumer's poll call
            log.debug("Not returning fetched records for partition {} since it is no longer assigned", tp);
        } else if (!subscriptions.isFetchable(tp)) {
            // this can happen when a partition is paused before fetched records are returned to the consumer's
            // poll call or if the offset is being reset.
            // It can also happen under the Consumer rebalance protocol, when the consumer changes its subscription.
            // Until the consumer receives an updated assignment from the coordinator, it can hold assigned partitions
            // that are not in the subscription anymore, so we make them not fetchable.
            log.debug("Not returning fetched records for assigned partition {} since it is no longer fetchable", tp);
        } else {
            SubscriptionState.FetchPosition position = subscriptions.position(tp);

            if (position == null)
                throw new IllegalStateException("Missing position for fetchable partition " + tp);

            if (nextInLineFetch.nextFetchOffset() == position.offset) {
                List<ConsumerRecord<K, V>> partRecords = nextInLineFetch.fetchRecords(fetchConfig,
                        deserializers,
                        maxRecords);

                log.trace("Returning {} fetched records at offset {} for assigned partition {}",
                        partRecords.size(), position, tp);

                boolean positionAdvanced = false;

                if (nextInLineFetch.nextFetchOffset() > position.offset) {
                    SubscriptionState.FetchPosition nextPosition = new SubscriptionState.FetchPosition(
                            nextInLineFetch.nextFetchOffset(),
                            nextInLineFetch.lastEpoch(),
                            position.currentLeader);
                    log.trace("Updating fetch position from {} to {} for partition {} and returning {} records from `poll()`",
                            position, nextPosition, tp, partRecords.size());

View on GitHub (pinned to c31c9215e1)

Solutions

Ensure every partition you assign manually is seek()'d (or has committed offsets) before the first poll().
In ConsumerRebalanceListener.onPartitionsAssigned, set positions for all newly assigned partitions before returning.
Upgrade kafka-clients to the latest patch release; several Missing position races were fixed historically.
Avoid calling assign() and subscribe() interchangeably on the same consumer instance.

Example fix

// before
consumer.assign(Collections.singleton(tp));
consumer.poll(Duration.ofMillis(500)); // IllegalStateException: no position

// after
consumer.assign(Collections.singleton(tp));
consumer.seek(tp, 0L); // or rely on committed offsets + auto.offset.reset
consumer.poll(Duration.ofMillis(500));

Defensive patterns

Strategy: try-catch

Validate before calling

// Before relying on a partition's position, confirm it is initialized.
// KafkaConsumer does not expose positionOrNull() directly, but you can probe safely:
Set<TopicPartition> assigned = consumer.assignment();
for (TopicPartition tp : assigned) {
    try {
        long pos = consumer.position(tp); // forces position resolution if needed
        // a valid long means position is set; absence would raise IllegalStateException internally
    } catch (Exception ignore) {
        // trigger an offset reset / seek to a known offset before poll()
        consumer.seek(tp, OffsetResetStrategy.EARLIEST.equals(policy) ? 0L : consumer.endOffsets(Collections.singleton(tp)).get(tp));
    }
}

Try / catch

try {
    ConsumerRecords<K,V> records = consumer.poll(Duration.ofMillis(500));
} catch (IllegalStateException e) {
    if (e.getMessage() != null && e.getMessage().startsWith("Missing position for fetchable partition")) {
        // FetchCollector.java:166: subscriptions.position(tp) returned null for a
        // fetchable, assigned partition. Usually a transient post-rebalance state.
        log.warn("Position missing after assignment; pausing and re-polling: {}", e.getMessage());
        consumer.pause(consumer.assignment());
        consumer.resume(consumer.assignment()); // re-trigger position resolution
    } else {
        throw e;
    }
}

Prevention

After onPartitionsAssigned, call consumer.position(tp) (or seek explicitly) before relying on poll to return records for that partition.
Avoid mixing manual seek() with auto offset reset in the same subscription; pick one strategy per consumer.
Do not mutate assignment (subscribe/unsubscribe/assign) concurrently with poll(); the position map can desync.
If seen repeatedly, file a bug — this path indicates an internal state-machine invariant violation, not a user config error.

When it happens

Trigger: Triggered at FetchCollector.java:166 when, after isAssigned(tp) and isFetchable(tp) both pass, the position lookup returns null. Usually appears after a rebalance where assignment was updated but seek/position was never set, or when log.records are returned for a partition whose offset reset was requested but not yet applied.

Common situations: Race between ConsumerRebalanceListener.onPartitionsAssigned (which seeks partitions) and the subsequent poll() that tries to read them; manual assignment via assign() without a following seek() or without auto.offset.reset; bugs triggered by mixing assign() and subscribe(); rapid subscribe/unsubscribe churn; down-level kafka-clients versions with known position-tracking fixes.

Related errors

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/552f2042eface27e.json. Report an issue: GitHub.