{"id":"5c49ae7446354e76","repo":"apache/kafka","slug":"failed-to-get-offsets-by-times-in-ms-5c49ae","errorCode":null,"errorMessage":"Failed to get offsets by times in {}ms","messagePattern":"Failed to get offsets by times in (.+?)ms","errorType":"exception","errorClass":"TimeoutException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/clients/consumer/internals/OffsetFetcher.java","lineNumber":192,"sourceCode":"\n            // if timeout is set to zero, do not try to poll the network client at all\n            // and return empty immediately; otherwise try to get the results synchronously\n            // and throw timeout exception if it cannot complete in time\n            if (timer.timeoutMs() == 0L)\n                return result;\n\n            client.poll(future, timer);\n\n            if (!future.isDone()) {\n                break;\n            } else if (remainingToSearch.isEmpty()) {\n                return result;\n            } else {\n                client.awaitMetadataUpdate(timer);\n            }\n        } while (timer.notExpired());\n\n        throw new TimeoutException(\"Failed to get offsets by times in \" + timer.elapsedMs() + \"ms\");\n    }\n\n    public Map<TopicPartition, Long> beginningOffsets(Collection<TopicPartition> partitions, Timer timer) {\n        return beginningOrEndOffset(partitions, ListOffsetsRequest.EARLIEST_TIMESTAMP, timer, false);\n    }\n\n    public Map<TopicPartition, Long> endOffsets(Collection<TopicPartition> partitions, Timer timer) {\n        return beginningOrEndOffset(partitions, ListOffsetsRequest.LATEST_TIMESTAMP, timer, false);\n    }\n\n    public OptionalLong currentLag(TopicPartition topicPartition) {\n        final Long lag = subscriptions.partitionLag(topicPartition, isolationLevel);\n\n        // if the log end offset is not known and hence cannot return lag and there is\n        // no in-flight list offset requested yet,\n        // issue a list offset request for that partition so that next time\n        // we may get the answer; we do not need to wait for the return value\n        // since we would not try to poll the network client synchronously","sourceCodeStart":174,"sourceCodeEnd":210,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/clients/consumer/internals/OffsetFetcher.java#L174-L210","documentation":"TimeoutException thrown by OffsetFetcher when offsetsForTimes (or listOffsets) cannot complete within the supplied timer. The retry loop polls the network client, awaits metadata updates between attempts, and if timer.notExpired() is false after exhausting attempts, the partial result is discarded and this exception is raised. The elapsed time is included so the caller can correlate against request timeouts.","triggerScenarios":"Thrown by KafkaConsumer.offsetsForTimes(), beginningOffsets(), or endOffsets() when the underlying ListOffsets requests do not all complete within the timeout passed to those calls. Triggered by slow brokers, leader elections mid-request, or repeated retried errors (e.g. NOT_LEADER_AVAILABLE) exhausting the timer.","commonSituations":"Broker overload or GC pauses, network latency, request.timeout.ms lower than the offsetsForTimes timeout, frequent leader elections,topic partition movement, or a thin client polling a large partition set. Also seen when the cluster is mid-recovery and metadata is unstable.","solutions":["Increase the timeout passed to offsetsForTimes / endOffsets / beginningOffsets (it is the method's Duration argument, not request.timeout.ms).","Raise request.timeout.ms and retry.backoff.ms so individual ListOffsets attempts do not fail-fast.","Verify cluster health: leader availability for the queried partitions, broker CPU/GC, and network latency with kafka-broker-api-versions.","Reduce the partition batch queried at once if a single slow partition dominates the timer."],"exampleFix":"// before\nMap<TopicPartition,OffsetAndTimestamp> r =\n    consumer.offsetsForTimes(targets, Duration.ofSeconds(1));\n\n// after\nMap<TopicPartition,OffsetAndTimestamp> r =\n    consumer.offsetsForTimes(targets, Duration.ofSeconds(30));","handlingStrategy":"retry","validationCode":"long requestTimeout = Math.max(timeoutMs, 2 * fetchMetadataTimeoutMs);\nif (requestTimeout < 1000L) {\n    throw new IllegalArgumentException(\"offsetsForTimes timeout must allow for a metadata round-trip; got \" + requestTimeout);\n}","typeGuard":null,"tryCatchPattern":"long backoff = 100L;\nfor (int attempt = 0; attempt < maxAttempts; attempt++) {\n    try {\n        return consumer.offsetsForTimes(timestampsToSearch, Duration.ofMillis(timeoutMs));\n    } catch (org.apache.kafka.common.errors.TimeoutException e) {\n        if (attempt == maxAttempts - 1) throw e;\n        try { Thread.sleep(backoff); } catch (InterruptedException ie) { Thread.currentThread().interrupt(); throw e; }\n        backoff = Math.min(backoff * 2, 5000L);\n    }\n}","preventionTips":["Size the timeout to at least the broker's advertised request timeout plus a network round-trip margin.","Ensure the broker(s) for the queried partitions are reachable and not in the middle of a leader election before issuing offsetsForTimes.","Warm up metadata (poll once, or await a metadata refresh) before bulk offsetsForTimes calls on fresh consumers.","When querying many partitions, batch them in a single call rather than N small calls to amortize metadata waits.","In containers with strict CPU throttling, raise the timeout — the consumer's own timer may be expiring under contention."],"tags":["kafka","consumer","timeout","offsets","metadata"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}