{"id":"552f2042eface27e","repo":"apache/kafka","slug":"missing-position-for-fetchable-partition","errorCode":null,"errorMessage":"Missing position for fetchable partition {}","messagePattern":"Missing position for fetchable partition (.+?)","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchCollector.java","lineNumber":166,"sourceCode":"\n    private Fetch<K, V> fetchRecords(final CompletedFetch nextInLineFetch, int maxRecords) {\n        final TopicPartition tp = nextInLineFetch.partition;\n\n        if (!subscriptions.isAssigned(tp)) {\n            // this can happen when a rebalance happened before fetched records are returned to the consumer's poll call\n            log.debug(\"Not returning fetched records for partition {} since it is no longer assigned\", tp);\n        } else if (!subscriptions.isFetchable(tp)) {\n            // this can happen when a partition is paused before fetched records are returned to the consumer's\n            // poll call or if the offset is being reset.\n            // It can also happen under the Consumer rebalance protocol, when the consumer changes its subscription.\n            // Until the consumer receives an updated assignment from the coordinator, it can hold assigned partitions\n            // that are not in the subscription anymore, so we make them not fetchable.\n            log.debug(\"Not returning fetched records for assigned partition {} since it is no longer fetchable\", tp);\n        } else {\n            SubscriptionState.FetchPosition position = subscriptions.position(tp);\n\n            if (position == null)\n                throw new IllegalStateException(\"Missing position for fetchable partition \" + tp);\n\n            if (nextInLineFetch.nextFetchOffset() == position.offset) {\n                List<ConsumerRecord<K, V>> partRecords = nextInLineFetch.fetchRecords(fetchConfig,\n                        deserializers,\n                        maxRecords);\n\n                log.trace(\"Returning {} fetched records at offset {} for assigned partition {}\",\n                        partRecords.size(), position, tp);\n\n                boolean positionAdvanced = false;\n\n                if (nextInLineFetch.nextFetchOffset() > position.offset) {\n                    SubscriptionState.FetchPosition nextPosition = new SubscriptionState.FetchPosition(\n                            nextInLineFetch.nextFetchOffset(),\n                            nextInLineFetch.lastEpoch(),\n                            position.currentLeader);\n                    log.trace(\"Updating fetch position from {} to {} for partition {} and returning {} records from `poll()`\",\n                            position, nextPosition, tp, partRecords.size());","sourceCodeStart":148,"sourceCodeEnd":184,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/clients/consumer/internals/FetchCollector.java#L148-L184","documentation":"IllegalStateException(\"Missing position for fetchable partition X\") thrown inside FetchCollector.fetchRecords when a partition is assigned and fetchable but subscriptions.position(tp) returns null. The consumer cannot determine the offset at which to consume returned records, indicating an internal state-machine inconsistency between assignment and position tracking rather than a normal application error.","triggerScenarios":"Triggered at FetchCollector.java:166 when, after isAssigned(tp) and isFetchable(tp) both pass, the position lookup returns null. Usually appears after a rebalance where assignment was updated but seek/position was never set, or when log.records are returned for a partition whose offset reset was requested but not yet applied.","commonSituations":"Race between ConsumerRebalanceListener.onPartitionsAssigned (which seeks partitions) and the subsequent poll() that tries to read them; manual assignment via assign() without a following seek() or without auto.offset.reset; bugs triggered by mixing assign() and subscribe(); rapid subscribe/unsubscribe churn; down-level kafka-clients versions with known position-tracking fixes.","solutions":["Ensure every partition you assign manually is seek()'d (or has committed offsets) before the first poll().","In ConsumerRebalanceListener.onPartitionsAssigned, set positions for all newly assigned partitions before returning.","Upgrade kafka-clients to the latest patch release; several Missing position races were fixed historically.","Avoid calling assign() and subscribe() interchangeably on the same consumer instance."],"exampleFix":"// before\nconsumer.assign(Collections.singleton(tp));\nconsumer.poll(Duration.ofMillis(500)); // IllegalStateException: no position\n\n// after\nconsumer.assign(Collections.singleton(tp));\nconsumer.seek(tp, 0L); // or rely on committed offsets + auto.offset.reset\nconsumer.poll(Duration.ofMillis(500));","handlingStrategy":"try-catch","validationCode":"// Before relying on a partition's position, confirm it is initialized.\n// KafkaConsumer does not expose positionOrNull() directly, but you can probe safely:\nSet<TopicPartition> assigned = consumer.assignment();\nfor (TopicPartition tp : assigned) {\n    try {\n        long pos = consumer.position(tp); // forces position resolution if needed\n        // a valid long means position is set; absence would raise IllegalStateException internally\n    } catch (Exception ignore) {\n        // trigger an offset reset / seek to a known offset before poll()\n        consumer.seek(tp, OffsetResetStrategy.EARLIEST.equals(policy) ? 0L : consumer.endOffsets(Collections.singleton(tp)).get(tp));\n    }\n}","typeGuard":null,"tryCatchPattern":"try {\n    ConsumerRecords<K,V> records = consumer.poll(Duration.ofMillis(500));\n} catch (IllegalStateException e) {\n    if (e.getMessage() != null && e.getMessage().startsWith(\"Missing position for fetchable partition\")) {\n        // FetchCollector.java:166: subscriptions.position(tp) returned null for a\n        // fetchable, assigned partition. Usually a transient post-rebalance state.\n        log.warn(\"Position missing after assignment; pausing and re-polling: {}\", e.getMessage());\n        consumer.pause(consumer.assignment());\n        consumer.resume(consumer.assignment()); // re-trigger position resolution\n    } else {\n        throw e;\n    }\n}","preventionTips":["After onPartitionsAssigned, call consumer.position(tp) (or seek explicitly) before relying on poll to return records for that partition.","Avoid mixing manual seek() with auto offset reset in the same subscription; pick one strategy per consumer.","Do not mutate assignment (subscribe/unsubscribe/assign) concurrently with poll(); the position map can desync.","If seen repeatedly, file a bug — this path indicates an internal state-machine invariant violation, not a user config error."],"tags":["consumer","rebalance","position","internal-error"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}