{"record":{"id":"d0111d180eab7db3","repo":"apache/cassandra","slug":"unable-to-obtain-private-address-for-node-from-clo","errorCode":null,"errorMessage":"Unable to obtain private address for node from cloud metadata service","messagePattern":"Unable to obtain private address for node from cloud metadata service","errorType":"validation","errorClass":"ConfigurationException","httpStatus":null,"severity":"error","filePath":"src/java/org/apache/cassandra/locator/Ec2MultiRegionAddressConfig.java","lineNumber":87,"sourceCode":"            if (DatabaseDescriptor.getBroadcastRpcAddress() == null)\n            {\n                logger.info(\"broadcast_rpc_address unset, broadcasting public IP as rpc_address: {}\", localPublicAddress);\n                DatabaseDescriptor.setBroadcastRpcAddress(broadcastAddress);\n            }\n        }\n        catch (UnknownHostException e)\n        {\n            throw new ConfigurationException(\"Unable to obtain public address for node from cloud metadata service\", e);\n        }\n\n        try\n        {\n            InetAddress privateAddress = InetAddress.getByName(localPrivateAddress);\n            FBUtilities.setLocalAddress(privateAddress);\n        }\n        catch (UnknownHostException e)\n        {\n            throw new ConfigurationException(\"Unable to obtain private address for node from cloud metadata service\", e);\n        }\n    }\n\n    @Override\n    public boolean preferLocalConnections()\n    {\n        // Always prefer re-connecting on private addresses if in the same datacenter (i.e. region)\n        return true;\n    }\n}\n","sourceCodeStart":69,"sourceCodeEnd":98,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/locator/Ec2MultiRegionAddressConfig.java#L69-L98","documentation":"After resolving the public address, Ec2MultiRegionAddressConfig sets the node's local (private) address by resolving localAddressAndPort from the EC2 metadata service. An UnknownHostException there is wrapped as this ConfigurationException, aborting startup because the node cannot determine its own private IP.","triggerScenarios":"configureAddresses() catches UnknownHostException from InetAddress.getByName(localPrivateAddress) — the metadata endpoint supplying local-ipv4/local hostname failed or returned an unresolvable value.","commonSituations":"IMDS blocked by iptables/security-group changes; instance metadata options disabled (HttpTokens/endpoint misconfig); IMDSv2 token failures making the metadata lookup return nothing; exotic networking (containers over EC2) where the metadata hostname doesn't resolve.","solutions":["Verify metadata access: curl -s http://169.254.169.254/latest/meta-data/local-ipv4 returns the private IP","Set listen_address explicitly in cassandra.yaml to bypass the metadata lookup for the private address","Re-enable instance metadata options (IMDSv2 endpoint reachable, hop limit >= 1, metadata enabled) on the EC2 instance","Check host-level firewall/NAT rules that might block the 169.254.169.254 link-local address"],"exampleFix":"// before (cassandra.yaml)\nlisten_address: # unset; resolved via EC2 metadata\n# after\nlisten_address: 10.0.4.17","handlingStrategy":"try-catch","validationCode":"String priv = shell(\"curl -s -m 2 http://169.254.169.254/latest/meta-data/local-ipv4\");\nif (priv == null || priv.isBlank()) throw new IllegalStateException(\"EC2 local-ipv4 metadata unavailable\");","typeGuard":null,"tryCatchPattern":"try { Ec2MultiRegionAddressConfig.configureAddresses(); }\ncatch (ConfigurationException e) { logger.error(\"Cannot resolve EC2 private address: {}\", e.getCause(), e); throw e; }","preventionTips":["Pin listen_address explicitly on EC2 nodes where possible","Include IMDS reachability in node bootstrap health checks","Keep instance metadata options enabled and hop limits sufficient (especially for containers)"],"tags":["ec2","network","metadata-service","dns","startup"],"backgroundTag":"http-request-failed","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-14T21:17:11.552Z"}