apache/kafka · error · TimeoutException

Operation timed out before completion

Error message

Operation timed out before completion

What it means

Thrown by ShareConsumerImpl when a background future (fetch, commit, or other share-consumer operation) does not complete before the request timer expires. The share consumer delegates work to an async background thread and polls a loop bounded by a Timer; when timer.notExpired() returns false before the future resolves, the loop exits and raises this TimeoutException. It is the share group equivalent of a poll/commit request that the broker or network never satisfied in time.

Source

Thrown at clients/src/main/java/org/apache/kafka/clients/consumer/internals/ShareConsumerImpl.java:1382

                } else if (!hadEvents) {
                    // If the above processing yielded no events, then let's sit tight for a bit to allow the
                    // background thread to either finish the task, or populate the background event
                    // queue with things to process in our next loop.
                    Timer pollInterval = time.timer(100L);
                    log.trace("Waiting {} ms for future {} to complete", pollInterval.remainingMs(), future);
                    T result = ConsumerUtils.getResult(future, pollInterval);
                    log.trace("Future {} completed successfully", future);
                    return result;
                }
            } catch (TimeoutException e) {
                // Ignore this as we will retry the event until the timeout expires.
            } finally {
                timer.update();
            }
        } while (timer.notExpired());

        log.trace("Future {} did not complete within timeout", future);
        throw new TimeoutException("Operation timed out before completion");
    }

    // Visible for testing
    void completeQuietly(final Utils.ThrowingRunnable function,
                         final String msg,
                         final AtomicReference<Throwable> firstException) {
        try {
            function.run();
        } catch (TimeoutException e) {
            log.debug("Timeout expired before the {} operation could complete.", msg);
        } catch (Exception e) {
            firstException.compareAndSet(null, e);
        }
    }

    @Override
    public String clientId() {
        return clientId;

View on GitHub (pinned to c31c9215e1)

Solutions

1. Increase request.timeout.ms and default.api.timeout.ms in the consumer config so the operation has time to complete.
2. Check broker reachability and share-group coordinator health (kafka-acls, broker logs, NetworkClient errors).
3. Ensure the broker version supports share groups and the topic is a regular (non-compacted) topic configured for share consumption.
4. Tune fetch.min.bytes / fetch.max.wait.ms so the broker returns sooner, and verify the background network thread is not blocked by other long-running calls.

Example fix

// before
props.put(ConsumerConfig.REQUEST_TIMEOUT_MS_CONFIG, 5000);
consumer.poll(Duration.ofSeconds(3));

// after
props.put(ConsumerConfig.REQUEST_TIMEOUT_MS_CONFIG, 30000);
props.put(ConsumerConfig.DEFAULT_API_TIMEOUT_MS_CONFIG, 60000);
consumer.poll(Duration.ofSeconds(30));

// No pre-check possible; the operation can only fail after the deadline elapses.
// Tune timeout via consumer config before constructing the consumer:
Properties props = new Properties();
props.put(ConsumerConfig.REQUEST_TIMEOUT_MS_CONFIG, 60000); // bump if broker is slow
props.put(ConsumerConfig.SESSION_TIMEOUT_MS_CONFIG, 60000);
// Pass a generous timeout to the poll/acknowledge call itself:
consumer.poll(Duration.ofSeconds(30));

try {
    consumer.poll(Duration.ofSeconds(30));
} catch (org.apache.kafka.common.errors.TimeoutException e) {
    // Transient: broker slow, no records, or background event not delivered in time.
    // Safe to retry the same call on the next loop iteration; no state is corrupted.
    log.debug("poll timed out, will retry", e);
}

When it happens

Trigger: Calling KafkaShareConsumer.poll(...), commitAcknowledgements(), or any blocking share-consumer API while the broker is unreachable, slow, or the background network thread is starved. Also triggered when the requested timeout (e.g. default.api.timeout.ms / request.timeout.ms) is shorter than the time the broker needs to acquire and deliver share-group records.

Common situations: Broker outage or network partition between client and broker; share-group coordinator under heavy load or rebalancing; request.timeout.ms set too low; DNS resolution stalls; GC pauses on client or broker; running against a broker version that does not yet support share groups (KIP-932), causing the request to hang until timeout.

Related errors

• Timeout of {}ms expired before the position for partition {}
• Timeout of {}ms expired before the last committed offset for
• Failed to get offsets by times in {}ms
• Operation timed out before completion
• Timeout of {}ms expired before successfully committing offse

AI-assisted analysis of apache/kafka@c31c9215e1 (2026-08-03). Data as JSON: /data/errors/2176ffb7deefd369.json. Report an issue: GitHub.