{"id":"974aa71108713b91","repo":"apache/kafka","slug":"timeout-of-ms-expired-before-the-position-for-pa","errorCode":null,"errorMessage":"Timeout of {}ms expired before the position for partition {} could be determined","messagePattern":"Timeout of (.+?)ms expired before the position for partition (.+?) could be determined","errorType":"exception","errorClass":"org.apache.kafka.common.errors.TimeoutException","httpStatus":null,"severity":"warning","filePath":"clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java","lineNumber":1249,"sourceCode":"    @Override\n    public long position(TopicPartition partition, Duration timeout) {\n        acquireAndEnsureOpen();\n        try {\n            if (!subscriptions.isAssigned(partition))\n                throw new IllegalStateException(\"You can only check the position for partitions assigned to this consumer.\");\n\n            Timer timer = time.timer(timeout);\n            do {\n                SubscriptionState.FetchPosition position = subscriptions.validPosition(partition);\n                if (position != null)\n                    return position.offset;\n\n                updateFetchPositions(timer);\n                timer.update();\n                wakeupTrigger.maybeTriggerWakeup();\n            } while (timer.notExpired());\n\n            throw new TimeoutException(\"Timeout of \" + timeout.toMillis() + \"ms expired before the position \" +\n                \"for partition \" + partition + \" could be determined\");\n        } finally {\n            release();\n        }\n    }\n\n    @Override\n    public Map<TopicPartition, OffsetAndMetadata> committed(final Set<TopicPartition> partitions) {\n        return committed(partitions, defaultApiTimeoutMs);\n    }\n\n    @Override\n    public Map<TopicPartition, OffsetAndMetadata> committed(final Set<TopicPartition> partitions,\n                                                            final Duration timeout) {\n        acquireAndEnsureOpen();\n        long start = time.nanoseconds();\n        try {\n            throwIfGroupIdNotDefined();","sourceCodeStart":1231,"sourceCodeEnd":1267,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java#L1231-L1267","documentation":"TimeoutException thrown by AsyncKafkaConsumer.position(TopicPartition, Duration) when the retry loop expires (timer.notExpired() returns false) before subscriptions.validPosition(partition) returns a non-null FetchPosition. position blocks updating fetch positions until either a valid position is materialized or the supplied timeout elapses; the message names the partition and the elapsed timeout in ms.","triggerScenarios":"Calling consumer.position(tp, timeout) with a short timeout before the consumer has resolved a fetch position for tp (e.g. immediately after assign/subscribe, before any poll has triggered a ListOffsets round-trip). Also triggered when the broker is slow to respond to ListOffsets, when the network is unhealthy, or when wakeups interrupt the loop.","commonSituations":"Tight timeout in tests calling position right after assign; broker under load or recovering; client disconnected/reconnecting during the call; auto.offset.reset=none combined with no committed offset forcing the consumer to wait; cold-start scenarios where the first position resolution requires multiple round-trips.","solutions":["Increase the timeout passed to position(partition, Duration.ofSeconds(N)) — give the broker time to resolve the offset.","Call consumer.poll(...) once before position so the assignment is materialized and the ListOffsets request has completed.","Raise default.api.timeout.ms / request.timeout.ms if the broker or network is consistently slow.","Investigate broker health / connectivity (bootstrap reachability, GC stalls, leader election) if timeouts persist."],"exampleFix":"// before\nconsumer.assign(List.of(tp));\nlong pos = consumer.position(tp, Duration.ofMillis(50)); // often too tight\n\n// after\nconsumer.assign(List.of(tp));\nconsumer.poll(Duration.ofMillis(100)); // materialize the position\nlong pos = consumer.position(tp, Duration.ofSeconds(5));","handlingStrategy":"retry","validationCode":"// No pre-check can fully prevent a timeout (it depends on broker responsiveness),\n// but you can size the timeout to your SLA and sanity-check the inputs.\nstatic long safePosition(Consumer<?, ?> c, TopicPartition tp, Duration timeout) {\n    if (timeout == null || timeout.isNegative() || timeout.isZero())\n        throw new IllegalArgumentException(\"position timeout must be positive, got \" + timeout);\n    if (!c.assignment().contains(tp))\n        throw new IllegalStateException(tp + \" is not assigned; cannot determine position\");\n    return c.position(tp, timeout);\n}\n\n// Heuristic: pick a timeout >= 2x your typical broker round-trip + rebalance time,\n// and never smaller than request.timeout.ms.","typeGuard":null,"tryCatchPattern":"// TimeoutException from position() is retriable if the underlying cause is broker\n// latency or an in-flight rebalance. Use bounded retries with backoff.\nDuration[] backoff = { Duration.ofMillis(100), Duration.ofMillis(500), Duration.ofSeconds(1) };\nlong offset = -1;\nfor (int attempt = 0; attempt <= backoff.length; attempt++) {\n    try {\n        offset = consumer.position(tp, Duration.ofSeconds(10));\n        break;\n    } catch (org.apache.kafka.common.errors.TimeoutException e) {\n        if (attempt == backoff.length)\n            throw new IllegalStateException(\"Could not determine position for \" + tp + \" after retries\", e);\n        log.warn(\"position({}) timed out (attempt {}); backing off {}\", tp, attempt + 1, backoff[attempt]);\n        try { Thread.sleep(backoff[attempt].toMillis()); } catch (InterruptedException ie) {\n            Thread.currentThread().interrupt();\n            throw new org.apache.kafka.common.errors.InterruptedException(ie);\n        }\n        // Triggering a poll often forces position resolution post-rebalance:\n        try { consumer.poll(Duration.ZERO); } catch (Exception ignored) {}\n    }\n}","preventionTips":["Call position() only after the consumer has completed its first poll() — before that, fetch positions are not yet established and will reliably time out.","Use the overloaded position(TopicPartition, Duration) and pass a timeout sized to your broker RTT, not the implicit default.api.timeout.ms.","Confirm the broker is reachable and not in a long GC / rebalance storm before treating a timeout as a code defect.","If timeouts recur, raise fetch.max.wait.ms / request.timeout.ms / default.api.timeout.ms in concert and monitor under-fetching partitions.","Prefer consumer.position(tp) over cached offsets for any logic that drives commit(); a stale cache makes the timeout worse, not better."],"tags":["consumer","position","timeout","network"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}