apache/cassandra · error · ConfigurationException

Unable to obtain private address for node from cloud metadat

Error message

Unable to obtain private address for node from cloud metadata service

What it means

After resolving the public address, Ec2MultiRegionAddressConfig sets the node's local (private) address by resolving localAddressAndPort from the EC2 metadata service. An UnknownHostException there is wrapped as this ConfigurationException, aborting startup because the node cannot determine its own private IP.

Source

Thrown at src/java/org/apache/cassandra/locator/Ec2MultiRegionAddressConfig.java:87

            if (DatabaseDescriptor.getBroadcastRpcAddress() == null)
            {
                logger.info("broadcast_rpc_address unset, broadcasting public IP as rpc_address: {}", localPublicAddress);
                DatabaseDescriptor.setBroadcastRpcAddress(broadcastAddress);
            }
        }
        catch (UnknownHostException e)
        {
            throw new ConfigurationException("Unable to obtain public address for node from cloud metadata service", e);
        }

        try
        {
            InetAddress privateAddress = InetAddress.getByName(localPrivateAddress);
            FBUtilities.setLocalAddress(privateAddress);
        }
        catch (UnknownHostException e)
        {
            throw new ConfigurationException("Unable to obtain private address for node from cloud metadata service", e);
        }
    }

    @Override
    public boolean preferLocalConnections()
    {
        // Always prefer re-connecting on private addresses if in the same datacenter (i.e. region)
        return true;
    }
}

View on GitHub (pinned to 88fd0f6a0e)

Solutions

1. Verify metadata access: curl -s http://169.254.169.254/latest/meta-data/local-ipv4 returns the private IP
2. Set listen_address explicitly in cassandra.yaml to bypass the metadata lookup for the private address
3. Re-enable instance metadata options (IMDSv2 endpoint reachable, hop limit >= 1, metadata enabled) on the EC2 instance
4. Check host-level firewall/NAT rules that might block the 169.254.169.254 link-local address

Example fix

// before (cassandra.yaml)
listen_address: # unset; resolved via EC2 metadata
# after
listen_address: 10.0.4.17

String priv = shell("curl -s -m 2 http://169.254.169.254/latest/meta-data/local-ipv4");
if (priv == null || priv.isBlank()) throw new IllegalStateException("EC2 local-ipv4 metadata unavailable");

try { Ec2MultiRegionAddressConfig.configureAddresses(); }
catch (ConfigurationException e) { logger.error("Cannot resolve EC2 private address: {}", e.getCause(), e); throw e; }

When it happens

Trigger: configureAddresses() catches UnknownHostException from InetAddress.getByName(localPrivateAddress) — the metadata endpoint supplying local-ipv4/local hostname failed or returned an unresolvable value.

Common situations: IMDS blocked by iptables/security-group changes; instance metadata options disabled (HttpTokens/endpoint misconfig); IMDSv2 token failures making the metadata lookup return nothing; exotic networking (containers over EC2) where the metadata hostname doesn't resolve.

Understand the failure class

Background: 'Something went wrong' / 'Request failed (500)' / 'HTTP error! status: 404' — what failed HTTP requests actually mean and how to find the real cause — this error's family across 28 libraries.

Related errors

• Unable to obtain public address for node from cloud metadata
• Replacement host name could not be resolved or scope_id was
• Configured ${configName} "${intf}" could not be found
• Configured ${configName} "${intf}" was found, but had no add
• Set listen_address OR listen_interface, not both

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/d0111d180eab7db3. Report an issue: GitHub.