{"id":"f25978aa29585160","repo":"apache/kafka","slug":"timeout-expired-after-timeoutms-ms-while-awaiting","errorCode":null,"errorMessage":"Timeout expired after {timeoutMs}ms while awaiting {operation}. {expectedTimeoutReason}","messagePattern":"Timeout expired after (.+?)ms while awaiting (.+?)\\. (.+?)","errorType":"exception","errorClass":"TimeoutException","httpStatus":null,"severity":"error","filePath":"clients/src/main/java/org/apache/kafka/clients/producer/internals/TransactionalRequestResult.java","lineNumber":54,"sourceCode":"    private TransactionalRequestResult(CountDownLatch latch, String operation) {\n        this.latch = latch;\n        this.operation = operation;\n    }\n\n    public void fail(RuntimeException error) {\n        this.error = error;\n        this.latch.countDown();\n    }\n\n    public void done() {\n        this.latch.countDown();\n    }\n\n    public void await(long timeout, TimeUnit unit, String expectedTimeoutReason) {\n        try {\n            boolean success = latch.await(timeout, unit);\n            if (!success) {\n                throw new TimeoutException(\"Timeout expired after \" + unit.toMillis(timeout) +\n                    \"ms while awaiting \" + operation + \". \" + expectedTimeoutReason);\n            }\n\n            isAcked = true;\n            if (error != null) {\n                throw error;\n            }\n        } catch (InterruptedException e) {\n            throw new InterruptException(\"Received interrupt while awaiting \" + operation, e);\n        }\n    }\n\n    public RuntimeException error() {\n        return error;\n    }\n\n    public boolean isSuccessful() {\n        return isCompleted() && error == null;","sourceCodeStart":36,"sourceCodeEnd":72,"githubUrl":"https://github.com/apache/kafka/blob/c31c9215e131f8c17e79f8901b48c13ee6aa8e7a/clients/src/main/java/org/apache/kafka/clients/producer/internals/TransactionalRequestResult.java#L36-L72","documentation":"Thrown by TransactionalRequestResult.await when the internal CountDownLatch does not reach zero within the configured timeout while waiting for a transactional control RPC (InitProducerId, AddPartitionsToTxn, EndTxn, TxnOffsetCommit, FindCoordinator, AlterProducerEpoch) to be acked by the coordinator. It surfaces as a TimeoutException carrying the elapsed ms, the operation name, and a broker-explained expectedTimeoutReason when the broker has already indicated the operation will take longer (e.g. CommitFailedException-style timing). The producer stays in the pending state and subsequent transactional calls will hit the pending-transition checks (errors 293/294).","triggerScenarios":"KafkaProducer.initTransactions/beginTransaction/commitTransaction/abortTransaction/sendOffsetsToTransaction internally calling TransactionalRequestResult.await and the underlying RPC not completing before the wait deadline. Typical when the coordinator (broker hosting __transaction_state partition for this transactional.id) is slow, unreachable, or rebalancing, or when the client network stalls.","commonSituations":"transaction.timeout.ms or request.timeout.ms too low; broker coordinator under load or recovering from a controller failover; misconfigured bootstrap servers or DNS issues causing FindCoordinator to hang; transaction.state.log under-replicated; long GC on broker or client; cross-AZ/region latency exceeding the wait; client behind a slow proxy or firewall doing deep packet inspection.","solutions":["Increase request.timeout.ms and delivery.timeout.ms (and broker transaction.max.timeout.ms) to exceed realistic RPC latency under peak load.","Verify coordinator health: which broker owns the __transaction_state partition for the transactional.id hash, check its load, GC, and replication status.","Check network path between client and brokers (latency, packet loss, firewall timeouts on idle connections).","If transient, retry the SAME transactional operation (it's pending — see errors 293/294); if persistent, recreate the producer after close()."],"exampleFix":"// before\nprops.put(ProducerConfig.TRANSACTION_TIMEOUT_CONFIG, \"30000\");  // 30s, too tight under load\nprops.put(ProducerConfig.DELIVERY_TIMEOUT_MS_CONFIG, \"60000\");\n\n// after\nprops.put(ProducerConfig.TRANSACTION_TIMEOUT_CONFIG, \"90000\");   // 90s\nprops.put(ProducerConfig.DELIVERY_TIMEOUT_MS_CONFIG, \"120000\");  // 120s\n// and on broker: transaction.max.timeout.ms >= 120000","handlingStrategy":"retry","validationCode":"// Pre-validate timeouts before issuing transactional calls so your own configured\n// delivery timeout is not shorter than the work you expect:\nlong delivery = Long.parseLong(props.getProperty(ProducerConfig.DELIVERY_TIMEOUT_MS_CONFIG, \"120000\"));\nlong txn     = Long.parseLong(props.getProperty(ProducerConfig.TRANSACTION_TIMEOUT_CONFIG, \"60000\"));\nif (delivery <= 0 || txn <= 0) throw new IllegalArgumentException(\"timeouts must be positive\");\nif (txn > 900000)              throw new IllegalArgumentException(\"txn timeout exceeds broker max\");","typeGuard":null,"tryCatchPattern":"try {\n    producer.commitTransaction();\n} catch (org.apache.kafka.common.errors.TimeoutException e) {\n    // Transactional control request did not finish in time. Idempotently retry the same\n    // method; the broker deduplicates transaction state transitions.\n    try {\n        producer.commitTransaction();\n    } catch (TimeoutException again) {\n        producer.close(Duration.ZERO);\n        throw new RuntimeException(\"Transactional commit timed out twice; broker unreachable\", again);\n    }\n}","preventionTips":["Set delivery.timeout.ms generously (>= request.timeout.ms + retry headroom) and ensure transaction.timeout.ms < broker transaction.max.timeout.ms.","Monitor the transaction coordinator's responsiveness; persistent timeouts usually indicate broker load or a partitioned coordinator.","Catch TimeoutException specifically (not just Exception) so you can retry transactional control calls idempotently while still surfacing genuine data errors."],"tags":["transactions","eos","timeout","coordinator"],"analyzedSha":"c31c9215e131f8c17e79f8901b48c13ee6aa8e7a","analyzedAt":"2026-08-03T12:34:05.770Z","schemaVersion":2}