{"record":{"id":"313f0ff7b14e24af","repo":"apache/cassandra","slug":"could-not-discover-cms-from","errorCode":null,"errorMessage":"Could not discover CMS from ","messagePattern":"Could not discover CMS from ","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"src/java/org/apache/cassandra/tcm/RemoteProcessor.java","lineNumber":416,"sourceCode":"            public void onResponse(Message<DiscoveredNodes> msg)\n            {\n                promise.setSuccess(msg.payload);\n            }\n\n            @Override\n            public void onFailure(InetAddressAndPort from, RequestFailure failureReason)\n            {\n                // \"success\" - this lets us just try the next one in cmsIter\n                promise.setSuccess(new DiscoveredNodes(Collections.emptySet(), DiscoveredNodes.Kind.KNOWN_PEERS));\n            }\n        });\n        try\n        {\n            return promise.get(DatabaseDescriptor.getCmsAwaitTimeout().to(TimeUnit.MILLISECONDS), TimeUnit.MILLISECONDS);\n        }\n        catch (Exception e)\n        {\n            logger.warn(\"Could not discover CMS from \" + ep, e);\n        }\n        return new DiscoveredNodes(Collections.emptySet(), DiscoveredNodes.Kind.KNOWN_PEERS);\n    }\n\n    public static class CandidateIterator extends AbstractIterator<InetAddressAndPort>\n    {\n        private final Deque<InetAddressAndPort> candidates;\n        private final Set<InetAddressAndPort> elements;\n        private final boolean checkLive;\n\n        @SuppressWarnings(\"resource\")\n        public CandidateIterator(Collection<InetAddressAndPort> initialContacts)\n        {\n            this(initialContacts, true);\n        }\n\n        @SuppressWarnings(\"resource\")\n        public CandidateIterator(Collection<InetAddressAndPort> initialContacts, boolean checkLive)","sourceCodeStart":398,"sourceCodeEnd":434,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/tcm/RemoteProcessor.java#L398-L434","documentation":"RemoteProcessor.tryDiscover attempts to discover the Cluster Metadata Service (CMS) by contacting remote nodes; when the discovery promise does not complete within DatabaseDescriptor.getCmsAwaitTimeout(), the timeout/exception is logged as a warning and an empty DiscoveredNodes with Kind.KNOWN_PEERS is returned so the caller (cms) can fall back to known peers. This is a degraded-discovery warning, not a fatal error.","triggerScenarios":"Calling ClusterMetadataService.discover() (via RemoteProcessor.tryDiscover) when remote candidates do not respond before the cms_await_timeout elapses; network partition, firewall, or the target CMS host being down while the node is booting or re-registering with the cluster.","commonSituations":"Node joining a cluster whose CMS host is stopped or unreachable; misconfigured broadcast/rpc addresses so discovery RPCs go to the wrong IP; cms_await_timeout set too low for slow networks; starting a node while rest of cluster is still down.","solutions":["Verify the CMS hosts are running and reachable (nodetool status, ping/check rpc_address and broadcast_address).","Increase cms_await_timeout in cassandra.yaml (or the corresponding system property) to allow slower discovery.","Confirm internode connectivity and TLS settings; check for firewall or wrong seed/address configuration.","If this recurs at every startup, check the system.cluster_metadata logs on candidates to confirm which node actually hosts the CMS and re-run CMS relocation if needed."],"exampleFix":"// cassandra.yaml\n// before\ncms_await_timeout: 10s\n// after\ncms_await_timeout: 60s","handlingStrategy":"retry","validationCode":"// Before relying on discovery, check CMS reachability\nif (!ClusterMetadataService.instance().isCurrent() && ClusterMetadataService.instance().isRunning()) {\n    // discovery should succeed; otherwise expect timeout fallback\n}","typeGuard":null,"tryCatchPattern":"// Discovery degrades gracefully; handle the empty/KNOWN_PEERS result\nDiscoveredNodes found = processor.tryDiscover(ep);\nif (found.kind() == DiscoveredNodes.Kind.KNOWN_PEERS && found.nodes().isEmpty()) {\n    logger.warn(\"CMS discovery timed out; retrying after connectivity check\");\n}","preventionTips":["Keep cms_await_timeout generously sized for your network.","Monitor CMS host health and ensure at least one candidate is always reachable.","Validate broadcast/rpc address configuration before rolling out nodes."],"tags":["network","timeout","cms-discovery","startup"],"backgroundTag":"request-timeout","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}