apache/kafka · error · TimeoutException

Timeout of {}ms expired before the position for partition {}

Error message

Timeout of {}ms expired before the position for partition {} could be determined

What it means

Thrown by KafkaConsumer.position(TopicPartition, Duration) when the timer expires before the consumer can resolve a valid fetch position (e.g. while waiting for offset reset, coordinator fetch, or metadata). The position is not yet known because the partition has not finished initialization within the supplied (or default) timeout. It indicates the broker/coordinator path was too slow or unreachable, not a logic error.

Source

Thrown at clients/src/main/java/org/apache/kafka/clients/consumer/internals/ClassicKafkaConsumer.java:889

    @Override
    public long position(TopicPartition partition, final Duration timeout) {
        acquireAndEnsureOpen();
        try {
            if (!this.subscriptions.isAssigned(partition))
                throw new IllegalStateException("You can only check the position for partitions assigned to this consumer.");

            Timer timer = time.timer(timeout);
            do {
                SubscriptionState.FetchPosition position = this.subscriptions.validPosition(partition);
                if (position != null)
                    return position.offset;

                updateFetchPositions(timer);
                client.poll(timer);
            } while (timer.notExpired());

            throw new TimeoutException("Timeout of " + timeout.toMillis() + "ms expired before the position " +
                    "for partition " + partition + " could be determined");
        } finally {
            release();
        }
    }

    @Override
    public Map<TopicPartition, OffsetAndMetadata> committed(final Set<TopicPartition> partitions) {
        return committed(partitions, Duration.ofMillis(defaultApiTimeoutMs));
    }

    @Override
    public Map<TopicPartition, OffsetAndMetadata> committed(final Set<TopicPartition> partitions, final Duration timeout) {
        acquireAndEnsureOpen();
        long start = time.nanoseconds();
        try {
            throwIfGroupIdNotDefined();
            final Map<TopicPartition, OffsetAndMetadata> offsets;

View on GitHub (pinned to c31c9215e1)

Solutions

1. Pass a larger timeout to position(tp, Duration.ofSeconds(30)) or raise default.api.timeout.ms.
2. Ensure auto.offset.reset is set (earliest/latest) so a missing committed offset can be resolved quickly.
3. Verify broker reachability and group coordinator health; check for ongoing rebalances or coordinator failover.
4. Poll once before calling position() so initialization progresses before the timed call.

Example fix

// before
long pos = consumer.position(tp); // uses default api timeout

// after
consumer.poll(Duration.ofSeconds(2)); // let assignment + reset initialize
long pos = consumer.position(tp, Duration.ofSeconds(30));

// Nothing to validate pre-call; the failure is broker/coordinator latency.
// Budget a generous timeout derived from your SLO, not the default:
Duration posTimeout = Duration.ofMillis(Math.max(defaultApiTimeoutMs, 30_000));
consumer.position(tp, posTimeout);

// Retry with exponential backoff and a per-attempt cap; position() is idempotent and safe to re-issue:
Duration[] backoff = {Duration.ofMillis(500), Duration.ofMillis(2_000), Duration.ofMillis(10_000)};
for (int attempt = 0; attempt < backoff.length + 1; attempt++) {
    try {
        return consumer.position(tp, Duration.ofSeconds(30));
    } catch (TimeoutException e) {
        if (attempt == backoff.length) throw e;
        Thread.sleep(backoff[attempt].toMillis());
    }
}
throw new IllegalStateException("unreachable");

When it happens

Trigger: Calling position() with a short Duration right after assign()/subscribe() with no committed offset and no offset-reset policy; broker slow to respond; consumer unable to reach the group coordinator; offset reset still in flight when the deadline lapses.

Common situations: First position() call in a freshly started consumer with default.api.timeout.ms too low for the environment; network latency or broker load; missing auto.offset.reset config on a topic with no committed offsets; coordinator leader election in progress.

Related errors

• Timeout of {}ms expired before the last committed offset for
• Timeout of {}ms expired before the position for partition {}
• Timeout of {}ms expired before the last committed offset for
• Failed to get offsets by times in {}ms
• Operation timed out before completion

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/af4634c7e6efcef4.json. Report an issue: GitHub.