{"record":{"id":"560593852bcae26e","repo":"redis/redis-py","slug":"failed-to-subscribe-to-cluster-nodes-failed-node","errorCode":null,"errorMessage":"Failed to subscribe to cluster nodes: {failed_nodes}","messagePattern":"Failed to subscribe to cluster nodes: (.+?)","errorType":"exception","errorClass":"ConnectionError","httpStatus":null,"severity":"error","filePath":"redis/keyspace_notifications.py","lineNumber":2164,"sourceCode":"            new_nodes = set(current_primaries.keys()) - set(self._node_pubsubs.keys())\n            failed_nodes: list[str] = []\n            for node_name in new_nodes:\n                node = current_primaries[node_name]\n                pubsub = self._ensure_node_pubsub(node)\n\n                try:\n                    if self._subscribed_patterns:\n                        pubsub.psubscribe(**self._subscribed_patterns)\n                    if self._subscribed_channels:\n                        pubsub.subscribe(**self._subscribed_channels)\n                except Exception:\n                    # Subscription failed - remove from dict so retry is possible\n                    self._cleanup_node(node_name)\n                    failed_nodes.append(node_name)\n\n            # Raise after attempting all nodes so we don't skip any\n            if failed_nodes:\n                raise ConnectionError(\n                    f\"Failed to subscribe to cluster nodes: {', '.join(failed_nodes)}\"\n                )\n\n    def close(self):\n        \"\"\"Close all pubsub connections and clean up resources.\"\"\"\n        self._closed = True\n        for node_name in list(self._node_pubsubs.keys()):\n            self._cleanup_node(node_name)\n        self._subscribed_patterns.clear()\n        self._subscribed_channels.clear()\n","sourceCodeStart":2146,"sourceCodeEnd":2175,"githubUrl":"https://github.com/redis/redis-py/blob/6a6b581b48225afa0b76912d1028c6035baee932/redis/keyspace_notifications.py#L2146-L2175","documentation":"During a topology refresh, ClusterKeyspaceNotifications (redis/keyspace_notifications.py:2164) tries to subscribe each newly-discovered primary node to the currently-registered patterns/channels. If the psubscribe/subscribe call for a node raises, that node is cleaned up, recorded in failed_nodes, and after all new nodes are attempted a ConnectionError listing the failed node names is raised. It is raised only when at least one node could not be (re)subscribed.","triggerScenarios":"A cluster failover/reshard adds new primaries while one or more of them are unreachable (network partition, still starting up, misconfigured), so the per-node psubscribe/subscribe throws during refresh. Also when a node rejects the pubsub connection (maxclients, auth).","commonSituations":"Right after a primary failover during which a node is briefly unavailable; partial network outage in multi-AZ cluster; rolling restart where nodes come back at different times; auth/ACL mismatch on a subset of nodes.","solutions":["Retry the operation: the next topology refresh will re-attempt subscription on the recovered node.","Verify all primary nodes are reachable and have consistent ACL/auth config.","Ensure the cluster client's connection settings (timeout, retry) tolerate transient node unavailability during refresh.","Check maxclients / node health for the named failed nodes."],"exampleFix":"// before\nckn.run_in_thread(poll_timeout=0.1)  # raises during failover\n// after\nimport time\nfrom redis.exceptions import ConnectionError\nfor _ in range(10):\n    try:\n        ckn.run_in_thread(poll_timeout=0.1); break\n    except ConnectionError:\n        time.sleep(1)  # wait for new primary to finish failover","handlingStrategy":"retry","validationCode":"def cluster_healthy(cluster) -> bool:\n    try:\n        primaries = cluster.get_primaries()\n        return all(cluster.ping(node) for node in primaries)\n    except Exception:\n        return False","typeGuard":null,"tryCatchPattern":"import time\nfrom redis.exceptions import ConnectionError\nfor _ in range(10):\n    try:\n        ckn.run_in_thread(poll_timeout=0.1)\n        break\n    except ConnectionError:\n        time.sleep(1)\nelse:\n    raise","preventionTips":["Retry on the next topology refresh once unreachable primaries recover.","Verify consistent ACL/auth across all cluster nodes.","Tune connection timeout/retry so transient node unavailability during refresh is tolerated.","Watch maxclients and node health for the named failed nodes."],"tags":["keyspace-notifications","cluster","connection","failover","topology"],"backgroundTag":null,"analyzedSha":"6a6b581b48225afa0b76912d1028c6035baee932","analyzedAt":"2026-08-10T12:52:44.840Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}