conductor-oss/conductor · critical · RuntimeException

Retries completed. Timeout while connecting to <addresses>

Error message

Retries completed. Timeout while connecting to <addresses>

What it means

createConnection caught TimeoutException, retrySettings is non-null, but retry.continueOrPropogate(e, retryIndex) threw, meaning the retry budget was exhausted while the handshake kept timing out. The original TimeoutException is wrapped as the cause.

Source

Thrown at amqp/src/main/java/com/netflix/conductor/contribs/queue/amqp/AMQPConnection.java:165

                if (retry == null) {
                    final String error =
                            "Timeout while connecting to "
                                    + Arrays.stream(addresses)
                                            .map(address -> address.toString())
                                            .collect(Collectors.joining(","));
                    LOGGER.error(error, e);
                    throw new RuntimeException(error, e);
                }
                try {
                    retry.continueOrPropogate(e, retryIndex);
                } catch (Exception ex) {
                    final String error =
                            "Retries completed. Timeout while connecting to "
                                    + Arrays.stream(addresses)
                                            .map(address -> address.toString())
                                            .collect(Collectors.joining(","));
                    LOGGER.error(error, e);
                    throw new RuntimeException(error, e);
                }
                retryIndex++;
            }
        }
    }

    public Channel getOrCreateChannel(ConnectionType connectionType, String queueOrExchangeName)
            throws Exception {
        LOGGER.debug(
                "Accessing the channel for queueOrExchange {} with type {} ",
                queueOrExchangeName,
                connectionType);
        switch (connectionType) {
            case SUBSCRIBER:
                String subChnName = connectionType + ";" + queueOrExchangeName;
                if (subscriberReservedChannelPool.containsKey(subChnName)) {
                    Channel locChn = subscriberReservedChannelPool.get(subChnName);
                    if (locChn != null && locChn.isOpen()) {

View on GitHub (pinned to cf7c3e4a8a)

Solutions

1. Investigate broker load and network latency (the timeout is being hit on every attempt, not just once).
2. Raise the connection timeout AND/OR the retry budget, depending on whether the problem is per-attempt latency or outage length.
3. Add a circuit breaker so repeated timeout exhaustion fails fast upstream.
4. Confirm there is not a firewall doing SYN-proxying that stalls the AMQP handshake.

Example fix

# before: every handshake attempt times out and retries exhaust immediately
conductor.amqp.connectionTimeoutInMilliSecs=2000
conductor.amqp.retry.limit=3

# after: widen both the per-attempt timeout and the retry budget
conductor.amqp.connectionTimeoutInMilliSecs=10000
conductor.amqp.retry.limit=6

// Handshake retries exhausted; this is a hard failure
try {
    amqpConnection.getOrCreateChannel(type, name);
} catch (RuntimeException e) {
    LOGGER.error("AMQP handshake retries exhausted: {}", e.getMessage(), e);
    throw e; // trip circuit breaker
}

When it happens

Trigger: factory.newConnection keeps throwing TimeoutException on every attempt until the AMQPRetryPattern limit/duration is reached, then continueOrPropogate raises and the retries-completed message is thrown.

Common situations: A persistently slow or overloaded broker that cannot complete the handshake within any single attempt's timeout across the whole retry window; a network path with sustained high latency or packet loss.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• Retries completed. IO error while connecting to <addresses>
• Timeout while connecting to <addresses>
• Failed to open connection
• IO error while connecting to <addresses>
• Retries completed. Cannot open <connType> channel on <addres

AI-assisted analysis of conductor-oss/conductor@cf7c3e4a8a (2026-08-14). Data as JSON: /api/errors/ec2d0d7937329cf2. Report an issue: GitHub.