{"record":{"id":"a444be8602856a99","repo":"conductor-oss/conductor","slug":"retries-completed-io-error-while-connecting-to-a","errorCode":null,"errorMessage":"Retries completed. IO error while connecting to <addresses>","messagePattern":"Retries completed\\. IO error while connecting to <addresses>","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"critical","filePath":"amqp/src/main/java/com/netflix/conductor/contribs/queue/amqp/AMQPConnection.java","lineNumber":142,"sourceCode":"                if (retry == null) {\n                    final String error =\n                            \"IO error while connecting to \"\n                                    + Arrays.stream(addresses)\n                                            .map(address -> address.toString())\n                                            .collect(Collectors.joining(\",\"));\n                    LOGGER.error(error, e);\n                    throw new RuntimeException(error, e);\n                }\n                try {\n                    retry.continueOrPropogate(e, retryIndex);\n                } catch (Exception ex) {\n                    final String error =\n                            \"Retries completed. IO error while connecting to \"\n                                    + Arrays.stream(addresses)\n                                            .map(address -> address.toString())\n                                            .collect(Collectors.joining(\",\"));\n                    LOGGER.error(error, e);\n                    throw new RuntimeException(error, e);\n                }\n                retryIndex++;\n            } catch (final TimeoutException e) {\n                AMQPRetryPattern retry = retrySettings;\n                if (retry == null) {\n                    final String error =\n                            \"Timeout while connecting to \"\n                                    + Arrays.stream(addresses)\n                                            .map(address -> address.toString())\n                                            .collect(Collectors.joining(\",\"));\n                    LOGGER.error(error, e);\n                    throw new RuntimeException(error, e);\n                }\n                try {\n                    retry.continueOrPropogate(e, retryIndex);\n                } catch (Exception ex) {\n                    final String error =\n                            \"Retries completed. Timeout while connecting to \"","sourceCodeStart":124,"sourceCodeEnd":160,"githubUrl":"https://github.com/conductor-oss/conductor/blob/cf7c3e4a8adfb158be778ab1ec525323c363cd3a/amqp/src/main/java/com/netflix/conductor/contribs/queue/amqp/AMQPConnection.java#L124-L160","documentation":"createConnection caught IOException, retrySettings is non-null, but retry.continueOrPropogate(e, retryIndex) itself threw, meaning the configured retry budget (attempt limit or duration) has been exhausted while the IOException keeps recurring. The original IOException is attached as the cause of the propagated RuntimeException.","triggerScenarios":"factory.newConnection keeps throwing IOException across every retry attempt until the AMQPRetryPattern limit/duration is reached, at which point continueOrPropogate raises and the retries-completed message is thrown.","commonSituations":"Broker down for longer than the configured retry window; a persistent authentication failure (non-transient, so retries never help); a sustained network partition; a misconfigured port that refuses every attempt.","solutions":["Confirm the broker is actually up and reachable for the full retry window (the outage outlasted the retries).","Fix the root cause first: if it is auth/vhost/network, retries cannot succeed and must not be lengthened blindly.","If the outage is legitimately transient but longer than expected, raise the retry limit and/or duration in AMQPEventQueueProperties.","Add a circuit breaker upstream so a long broker outage fails fast instead of burning the whole retry budget per call."],"exampleFix":"# before: retries exhaust during a multi-minute broker restart\nconductor.amqp.retry.limit=3\nconductor.amqp.retry.duration=1s\n\n# after: size the budget to the expected restart window\nconductor.amqp.retry.limit=10\nconductor.amqp.retry.duration=5s","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"// Retries already exhausted inside the library; treat as a hard failure\ntry {\n    amqpConnection.getOrCreateChannel(type, name);\n} catch (RuntimeException e) {\n    LOGGER.error(\"AMQP connection retries exhausted: {}\", e.getMessage(), e);\n    // trip a circuit breaker; fail the workflow operation rather than loop\n    throw e;\n}","preventionTips":["Size the AMQPRetryPattern limit/duration to the broker's realistic restart window.","Add a circuit breaker upstream so a long outage fails fast instead of exhausting retries per call.","Alert on repeated 'Retries completed' messages as a broker-health signal."],"tags":["amqp","rabbitmq","connection","retry","io"],"backgroundTag":null,"analyzedSha":"cf7c3e4a8adfb158be778ab1ec525323c363cd3a","analyzedAt":"2026-08-14T03:33:19.897Z","schemaVersion":2},"datasetVersion":"2026-08-14T05:17:29.042Z"}