{"record":{"id":"c86bcacd5ceb743c","repo":"conductor-oss/conductor","slug":"failed-to-open-connection","errorCode":null,"errorMessage":"Failed to open connection","messagePattern":"Failed to open connection","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"critical","filePath":"amqp/src/main/java/com/netflix/conductor/contribs/queue/amqp/AMQPConnection.java","lineNumber":91,"sourceCode":"\n    // Exposed for UT\n    public static void setAMQPConnection(AMQPConnection amqpConnection) {\n        AMQPConnection.amqpConnection = amqpConnection;\n    }\n\n    public Address[] getAddresses() {\n        return addresses;\n    }\n\n    private Connection createConnection(String connectionPrefix) {\n        int retryIndex = 1;\n        while (true) {\n            try {\n                Connection connection =\n                        factory.newConnection(\n                                addresses, System.getenv(\"HOSTNAME\") + \"-\" + connectionPrefix);\n                if (connection == null || !connection.isOpen()) {\n                    throw new RuntimeException(\"Failed to open connection\");\n                }\n                connection.addShutdownListener(\n                        new ShutdownListener() {\n                            @Override\n                            public void shutdownCompleted(ShutdownSignalException cause) {\n                                LOGGER.error(\n                                        \"Received a shutdown exception for the connection {}. reason {} cause{}\",\n                                        connection.getClientProvidedName(),\n                                        cause.getMessage(),\n                                        cause);\n                            }\n                        });\n                connection.addBlockedListener(\n                        new BlockedListener() {\n                            @Override\n                            public void handleUnblocked() throws IOException {\n                                LOGGER.info(\n                                        \"Connection {} is unblocked\",","sourceCodeStart":73,"sourceCodeEnd":109,"githubUrl":"https://github.com/conductor-oss/conductor/blob/cf7c3e4a8adfb158be778ab1ec525323c363cd3a/amqp/src/main/java/com/netflix/conductor/contribs/queue/amqp/AMQPConnection.java#L73-L109","documentation":"Thrown by AMQPConnection.createConnection after factory.newConnection(addresses, clientName) returns a Connection whose isOpen() is false (or, rarely, null). It is a post-handshake sanity check: the TCP/AMQP handshake completed but the resulting Connection is not usable because the broker closed it immediately afterwards. The rabbitmq-client almost never returns null, so the realistic trigger is isOpen()==false.","triggerScenarios":"First call to getOrCreateChannel(SUBSCRIBER|PUBLISHER, name) when the cached connection is null or closed, so createConnection runs and newConnection returns a connection that is already closed. Also on re-establishment after the cached publisher/subscriber connection goes down.","commonSituations":"Broker is under a memory/disk resource alarm and drops the new connection right after the handshake; broker is at the max_connections limit; vhost/user permission causes the broker to tear the connection down post-auth; a TLS mismatch causes a silent close before the channel layer sees it.","solutions":["Check broker resource alarms and limits: run `rabbitmqctl status` and verify vm_memory_high_watermark and disk free space are not triggered.","Verify the configured vhost exists and the user has access: `rabbitmqctl list_permissions -p <vhost>`.","Confirm the broker is not at its max_connections limit for the user/vhost.","Retry the queue operation: getOrCreateChannel recreates a closed connection on the next call, so transient broker pressure can self-heal.","If TLS is enabled, confirm the client truststore and protocol match the broker to avoid a post-handshake close."],"exampleFix":"# before: broker under memory pressure drops new connections\n# rabbitmq.conf\nvm_memory_high_watermark.relative = 0.4\n\n# after: raise the watermark (or add memory) so new connections survive the handshake\nvm_memory_high_watermark.relative = 0.7","handlingStrategy":"try-catch","validationCode":"// Pre-flight: confirm the broker is reachable and the factory can open a test connection\nConnectionFactory probe = new ConnectionFactory();\nprobe.setUri(\"amqp://\" + username + \"@\" + host + \":\" + port + \"/\" + vhost);\ntry (Connection c = probe.newConnection()) {\n    if (!c.isOpen()) throw new IllegalStateException(\"broker opened a closed connection\");\n}","typeGuard":null,"tryCatchPattern":"// getOrCreateChannel(ConnectionType, String) declares `throws Exception`\ntry {\n    Channel chn = amqpConnection.getOrCreateChannel(ConnectionType.PUBLISHER, queueName);\n    // ... use channel\n} catch (RuntimeException e) {\n    // 'Failed to open connection' surfaces here; log, back off, and rethrow or circuit-break\n    LOGGER.error(\"AMQP connection unavailable\", e);\n    throw e;\n}","preventionTips":["Monitor broker memory and disk watermarks so resource alarms never drop fresh connections.","Run a connectivity/health probe against the broker during application startup before the queue layer is wired.","Keep broker max_connections sized with headroom above expected publisher+subscriber counts."],"tags":["amqp","rabbitmq","connection","network"],"backgroundTag":null,"analyzedSha":"cf7c3e4a8adfb158be778ab1ec525323c363cd3a","analyzedAt":"2026-08-14T03:33:19.897Z","schemaVersion":2},"datasetVersion":"2026-08-14T05:17:29.042Z"}