kubernetes/kops · error

fetching metadata from '%s' returned status code '%d'

Error message

fetching metadata from '%s' returned status code '%d'

What it means

getFromMetadataService (upup/pkg/fi/cloudup/openstack/openstackmetadata/metadata.go:150) performs an HTTP GET against mds.serviceURL (the OpenStack metadata service). Only HTTP 200 is treated as success; any other status code produces this error including the URL and code.

Source

Thrown at upup/pkg/fi/cloudup/openstack/openstackmetadata/metadata.go:150

	return mds.parseMetadata(f)
}

// getFromMetadataService tries to get metadata from a metadata service endpoint and returns it as InstanceMetadata.
// If the service endpoint cannot be contacted or reports a different status than StatusOK it will return an error.
func (mds MetadataService) getFromMetadataService() (*InstanceMetadata, error) {
	var client http.Client

	resp, err := client.Get(mds.serviceURL)
	if err != nil {
		return nil, err
	}
	defer resp.Body.Close()

	if resp.StatusCode == http.StatusOK {
		return mds.parseMetadata(resp.Body)
	}

	err = fmt.Errorf("fetching metadata from '%s' returned status code '%d'", mds.serviceURL, resp.StatusCode)
	return nil, err
}

// parseMetadata reads JSON data from a Reader and returns it as InstanceMetadata.
func (mds MetadataService) parseMetadata(r io.Reader) (*InstanceMetadata, error) {
	var meta InstanceMetadata

	data, err := io.ReadAll(r)
	if err != nil {
		return nil, err
	}
	err = json.Unmarshal(data, &meta)
	if err != nil {
		return nil, err
	}

	return &meta, nil
}

View on GitHub (pinned to 4c8573c808)

Solutions

1. Check the exact status code in the message and correlate with the metadata service logs (nova-api-metadata).
2. Verify the instance network routes 169.254.169.254 to the metadata service (no interfering proxy, correct iptables/ dnsmasq rules).
3. Retry the request — 429/5xx are often transient; getMetadata may already retry via search order.
4. Fall back to config-drive metadata by configuring the search order to include ConfigDriveID.

Example fix

// before
if resp.StatusCode == http.StatusOK {
    return mds.parseMetadata(resp.Body)
}
err = fmt.Errorf("fetching metadata from '%s' returned status code '%d'", mds.serviceURL, resp.StatusCode)
// after — include the body for diagnosis
if resp.StatusCode == http.StatusOK {
    return mds.parseMetadata(resp.Body)
}
body, _ := io.ReadAll(io.LimitReader(resp.Body, 512))
err = fmt.Errorf("fetching metadata from '%s' returned status code '%d': %s", mds.serviceURL, resp.StatusCode, string(body))

// Probe the metadata service before relying on it
resp, err := http.Get("http://169.254.169.254/openstack/latest/meta_data.json")
if err != nil {
    return errors.New("metadata service unreachable at 169.254.169.254")
}
if resp.StatusCode != http.StatusOK {
    return fmt.Errorf("metadata service returned %d; check nova-api-metadata health", resp.StatusCode)
}

meta, err := mds.getFromMetadataService()
if err != nil {
    var statusErr interface{ Error() string }
    if errors.As(err, &statusErr) && strings.Contains(err.Error(), "status code '5") {
        // 5xx from metadata service — retry with backoff
        time.Sleep(2 * time.Second)
        return mds.getFromMetadataService()
    }
    return nil, err
}

When it happens

Trigger: The HTTP request to the metadata service completes but returns a status other than 200 — 404 when the route is absent, 401/403 on gated metadata, 429 on throttling, or 5xx from the nova-api-metadata side.

Common situations: Instance not on a network with the metadata service route (169.254.169.254 redirected incorrectly); a proxy intercepting metadata requests and returning an error page; overloaded nova-metadata returning 500s; older clouds lacking a requested metadata path.

Related errors

• failed to get private networks from hetzner cloud metadata:
• downloading %q: %w
• error reading tag file %q: %v
• failed to get droplet region: %s
• failed to get metadata URL %s: %v

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/61021d850bd7cf96. Report an issue: GitHub.