kubernetes/kops · error

failed to get private networks from hetzner cloud metadata:

Error message

failed to get private networks from hetzner cloud metadata: %w

What it means

During nodeup bootstrap, GetMetadataLocalIP resolves the node's internal IP. For Hetzner it calls the hcloud metadata service's PrivateNetworks endpoint; if that HTTP call fails (network error, non-200 response, timeout, or metadata service unavailable), the error is wrapped with this message and aborts the build.

Source

Thrown at nodeup/pkg/model/context.go:608

	switch c.BootConfig.CloudProvider {
	case kops.CloudProviderAWS:
		config, err := awsconfig.LoadDefaultConfig(ctx)
		if err != nil {
			return "", fmt.Errorf("failed to load AWS config: %w", err)
		}
		metadata := imds.NewFromConfig(config)
		localIPv4, err := getMetadata(ctx, metadata, "local-ipv4")
		if err != nil {
			return "", fmt.Errorf("failed to get local-ipv4 address from ec2 metadata: %w", err)
		}
		internalIP = localIPv4

	case kops.CloudProviderHetzner:
		client := hcloudmetadata.NewClient()
		privateNetworksYaml, err := client.PrivateNetworks()
		if err != nil {
			return "", fmt.Errorf("failed to get private networks from hetzner cloud metadata: %w", err)
		}
		var privateNetworks []struct {
			IP           net.IP   `json:"ip"`
			AliasIPs     []net.IP `json:"alias_ips"`
			InterfaceNum int      `json:"interface_num"`
			MACAddress   string   `json:"mac_address"`
			NetworkID    int      `json:"network_id"`
			NetworkName  string   `json:"network_name"`
			Network      string   `json:"network"`
			Subnet       string   `json:"subnet"`
			Gateway      net.IP   `json:"gateway"`
		}
		err = yaml.Unmarshal([]byte(privateNetworksYaml), &privateNetworks)
		if err != nil {
			return "", fmt.Errorf("failed to convert private networks to object: %w", err)
		}
		for _, privateNetwork := range privateNetworks {
			if privateNetwork.InterfaceNum == 1 {

View on GitHub (pinned to 4c8573c808)

Solutions

  1. Verify the Hetzner server has metadata service access: curl http://169.254.169.254/hetzner/v1/metadata/private-networks from the node
  2. Attach the server to a Hetzner private network if the cluster config requires one
  3. Check for firewall/ebtables rules blocking the 169.254.169.254 link-local address
  4. Re-run nodeup; the failure may be a transient metadata service outage

Example fix

// verify metadata reachability before diagnosing kOps
kubectl exec -it node -- curl -sf http://169.254.169.254/hetzner/v1/metadata/private-networks || echo 'metadata service unreachable'
Defensive patterns

Strategy: retry

Validate before calling

curl -sf http://169.254.169.254/hetzner/v1/metadata/private-networks >/dev/null && echo OK || echo UNREACHABLE

Try / catch

ip, err := ctx.GetMetadataLocalIP(nodeCtx)
if err != nil {
	if strings.Contains(err.Error(), "failed to get private networks from hetzner cloud metadata") {
		// retry with backoff; metadata service may be transiently down
		for i := 0; i < 3; i++ {
			if ip, err = ctx.GetMetadataLocalIP(nodeCtx); err == nil {
				break
			}
			time.Sleep(time.Duration(1<<i) * time.Second)
		}
	}
	if err != nil {
		return fmt.Errorf("node bootstrap aborted, metadata unavailable: %w", err)
	}
}

Prevention

When it happens

Trigger: nodeup runs on a Hetzner Cloud server but the hcloud metadata endpoint (http://169.254.169.254/hetzner/v1/metadata/private-networks) is unreachable, returns an HTTP error, or times out.

Common situations: Server attached to no private network while one is required; metadata service blocked by firewall/ebtables rules; transient Hetzner network outage during node bootstrap; running nodeup outside a real Hetzner instance (e.g. container/CI) with CloudProvider=hetzner.

Related errors


AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/76ad31e9fa85880c. Report an issue: GitHub.