{"record":{"id":"74f91f7d62b798d6","repo":"risingwavelabs/risingwave","slug":"cannot-open-client-to-compute-node-addr","errorCode":null,"errorMessage":"Cannot open client to compute node {addr:?}","messagePattern":"Cannot open client to compute node (.+?)","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"error","filePath":"src/ctl/src/cmd_impl/hummock/resize_cache.rs","lineNumber":53,"sourceCode":"    meta_cache_capacity: Option<u64>,\n    data_cache_capacity: Option<u64>,\n    clear_meta_cache: bool,\n    clear_data_cache: bool,\n) -> anyhow::Result<()> {\n    let meta_client = context.meta_client().await?;\n\n    let GetClusterInfoResponse { worker_nodes, .. } = match meta_client.get_cluster_info().await {\n        Ok(resp) => resp,\n        Err(e) => {\n            fail!(\"Failed to get cluster info: {}\", e.as_report());\n        }\n    };\n\n    let futures = worker_nodes.iter().map(|worker| async {\n        let addr = worker.get_host().expect(\"worker host must be set\");\n        let client = ComputeClient::new(addr.into(), &RpcClientConfig::default())\n            .await\n            .unwrap_or_else(|_| panic!(\"Cannot open client to compute node {addr:?}\"));\n        client\n            .resize_cache(ResizeCacheRequest {\n                meta_cache_capacity: meta_cache_capacity.unwrap_or(0),\n                data_cache_capacity: data_cache_capacity.unwrap_or(0),\n                clear_meta_cache,\n                clear_data_cache,\n            })\n            .await\n    });\n\n    if let Err(e) = try_join_all(futures).await {\n        fail!(\"Failed to resize cache: {}\", e.as_report())\n    }\n\n    Ok(())\n}\n","sourceCodeStart":35,"sourceCodeEnd":70,"githubUrl":"https://github.com/risingwavelabs/risingwave/blob/6469eb736d691e8e9b8a419a57edd6429ca77417/src/ctl/src/cmd_impl/hummock/resize_cache.rs#L35-L70","documentation":"The resize_cache ctl command connects to every worker (compute) node to issue a ResizeCache RPC. If establishing the gRPC client to a node fails, it panics with this message instead of continuing, because resizing requires every compute node to be reachable.","triggerScenarios":"Compute node is down, wrong host/port in the worker's registered host address, network partition, or TLS config mismatch when ComputeClient::new is called during resize_cache.","commonSituations":"Running resize against a cluster where a compute node crashed or was decommissioned but still registered in meta; firewall blocking the node's RPC port; DNS resolution failure in container environments.","solutions":["Verify the failing compute node is running and healthy (check its logs / process)","Compare the registered host address with the node's actual reachable address; fix worker registration or DNS","Ensure the RPC port is reachable from where you run the ctl command (firewall/security group)","Restart or re-register the stale worker node, then rerun resize_cache"],"exampleFix":"null","handlingStrategy":"retry","validationCode":"for addr in worker_addrs {\n    if tokio::net::TcpStream::connect(addr).await.is_err() {\n        eprintln!(\"compute node {} unreachable; fix before resize\", addr);\n    }\n}","typeGuard":null,"tryCatchPattern":"match ComputeClient::new(addr.into(), &RpcClientConfig::default()).await {\n    Ok(client) => { /* resize */ }\n    Err(e) => eprintln!(\"skipping node {addr:?}: {e}; ensure node is up and port reachable\"),\n}","preventionTips":["Verify all worker nodes are healthy before cluster-wide operations","Cross-check registered host addresses against actual network endpoints","Open firewall/security-group paths for compute RPC ports"],"tags":["network","grpc","compute-node","panic"],"backgroundTag":"connection-refused","analyzedSha":"6469eb736d691e8e9b8a419a57edd6429ca77417","analyzedAt":"2026-09-11T21:06:21.487Z","contentChangedAt":"2026-09-11T21:06:21.487Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}