{"id":"c5834398c437c2c5","repo":"apache/kafka","slug":"operation-timed-out-before-completion","errorCode":null,"errorMessage":"Operation timed out before completion","messagePattern":"Operation timed out before completion","errorType":"exception","errorClass":"org.apache.kafka.common.errors.TimeoutException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java","lineNumber":2479,"sourceCode":"                    // If the event is done (either successfully or otherwise), go ahead and attempt to return\n                    // without waiting. We use the ConsumerUtils.getResult() method here to handle the conversion\n                    // of the exception types.\n                    return ConsumerUtils.getResult(future);\n                } else if (!hadEvents) {\n                    // If the above processing yielded no events, then let's sit tight for a bit to allow the\n                    // background thread to either finish the task, or populate the background event\n                    // queue with things to process in our next loop.\n                    Timer pollInterval = time.timer(100L);\n                    return ConsumerUtils.getResult(future, pollInterval);\n                }\n            } catch (TimeoutException swallow) {\n                // Ignore this as we will retry the event until the timeout expires.\n            } finally {\n                timer.update();\n            }\n        } while (timer.notExpired());\n\n        throw new TimeoutException(\"Operation timed out before completion\");\n    }\n\n    static ConsumerRebalanceListenerCallbackCompletedEvent invokeRebalanceCallbacks(ConsumerRebalanceListenerInvoker rebalanceListenerInvoker,\n                                                                                    ConsumerRebalanceListenerMethodName methodName,\n                                                                                    SortedSet<TopicPartition> partitions,\n                                                                                    CompletableFuture<Void> future) {\n        Exception e;\n\n        try {\n            switch (methodName) {\n                case ON_PARTITIONS_REVOKED:\n                    e = rebalanceListenerInvoker.invokePartitionsRevoked(partitions);\n                    break;\n\n                case ON_PARTITIONS_ASSIGNED:\n                    e = rebalanceListenerInvoker.invokePartitionsAssigned(partitions);\n                    break;\n","sourceCodeStart":2461,"sourceCodeEnd":2497,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/clients/consumer/internals/AsyncKafkaConsumer.java#L2461-L2497","documentation":"Thrown by AsyncKafkaConsumer.processBackgroundEvents as a TimeoutException when the overall timer expires before the enqueued background event completes its CompletableFuture. The async consumer drives work on a background network thread and the application thread polls for completion; if the broker, coordinator, or callback chain does not finish within the operation's deadline (default.api.timeout.ms or the request-specific timer), this generic timeout fires. The message is intentionally generic because the same loop backs poll(), commitSync, position(), unsubscribe and other blocking calls.","triggerScenarios":"Any blocking call on the async consumer whose Future does not complete within the timer: commitSync() exceeding default.api.timeout.ms; poll() against an unavailable broker or during a long rebalance; unsubscribe() that cannot finish because a ConsumerRebalanceListener.onPartitionsRevoked callback on the application thread is itself blocking; position(partition) when offset fetch is slow.","commonSituations":"Broker outage or network partition making the coordinator unreachable; max.poll.interval.ms exceeded so the consumer was kicked out of the group mid-operation; a user-supplied ConsumerRebalanceListener that blocks (DB locks, slow HTTP) and stalls the handoff between background and application threads; default.api.timeout.ms set too low for a slow cluster; GC pauses or overloaded hosts stretching coordinator responses past the deadline.","solutions":["Increase default.api.timeout.ms and/or request.timeout.ms to values that match your cluster's observed latency, and verify the broker is healthy.","Inspect the consumer logs for the underlying cause: TimeoutException here is a wrapper; look for the preceding RebalanceInProgressException, NotCoordinatorException, or callback exceptions in the same trace.","Ensure any ConsumerRebalanceListener callbacks return quickly (no blocking I/O); offload slow work to a separate thread and only touch consumer state from the callback thread.","Tune retry.backoff.ms / retry.backoff.max.ms so transient coordinator movement recovers within the deadline.","If the issue is during rebalance, check max.poll.interval.ms versus your record-processing time and lower batch size or raise the interval."],"exampleFix":"// before\nprops.put(ConsumerConfig.DEFAULT_API_TIMEOUT_MS_CONFIG, 10000);\nconsumer.commitSync(); // throws TimeoutException on slow cluster\n\n// after\nprops.put(ConsumerConfig.DEFAULT_API_TIMEOUT_MS_CONFIG, 60000);\nprops.put(ConsumerConfig.REQUEST_TIMEOUT_MS_CONFIG, 30000);\nconsumer.commitSync();","handlingStrategy":"retry","validationCode":"// No pre-validation prevents a runtime timeout, but you can pre-size timeouts.\nProperties p = new Properties();\np.put(ConsumerConfig.DEFAULT_API_TIMEOUT_MS_CONFIG, 60000); // default ~60s\np.put(ConsumerConfig.REQUEST_TIMEOUT_MS_CONFIG, 30000);\np.put(ConsumerConfig.RETRY_BACKOFF_MS_CONFIG, 500);\np.put(ConsumerConfig.RETRY_BACKOFF_MAX_MS_CONFIG, 10000);","typeGuard":null,"tryCatchPattern":"// Retry with bounded attempts + exponential backoff for TimeoutException.\nint attempts = 0, maxAttempts = 3;\nwhile (true) {\n    try {\n        return consumer.position(partition); // or whichever op timed out\n    } catch (TimeoutException e) {\n        if (++attempts > maxAttempts) throw e;\n        long backoff = Math.min(1000L * (1L << attempts), 8000L);\n        Thread.sleep(backoff);\n    }\n}","preventionTips":["Set default.api.timeout.ms generously for slow brokers or cold-start metadata fetches.","Ensure brokers are reachable (network/bootstraps) before issuing operations; most timeouts are connectivity, not logic.","Keep the consumer's background event loop unblocked: do not run long logic inside ConsumerRebalanceListener callbacks (AsyncKafkaConsumer).","Distinguish retriable timeouts (broker down) from fatal ones (operation not supported) before retrying.","Expose timeout as a tunable in your own config so ops can widen it without code changes."],"tags":["kafka","consumer","async-consumer","timeout","network","rebalance"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}