kubernetes/kops · critical

querying attested document: %w

Error message

querying attested document: %w

What it means

CreateToken wraps failures from queryIMDSAttestedDocument — the GET of /metadata/attested/document (PKCS7-signed document containing a nonce) — with the "querying attested document" prefix. The attested document proves to the control plane that this token was minted on the actual Azure VM, so failure aborts bootstrap token creation.

Source

Thrown at upup/pkg/fi/cloudup/azure/azuremetadata/authenticator.go:64

	// bootstrap.Authenticator.CreateToken carries no context; the IMDS HTTP client's own timeout
	// bounds these calls.
	ctx := context.TODO()

	// Query IMDS for the VM's resource ID.
	metadata, err := QueryComputeInstanceMetadata(ctx)
	if err != nil {
		return "", fmt.Errorf("querying instance metadata: %w", err)
	}
	if metadata.ResourceID == "" {
		return "", fmt.Errorf("missing resource ID")
	}
	klog.V(4).Infof("Azure authenticator obtained resource ID %q", metadata.ResourceID)

	// Query IMDS for a PKCS7-signed attested document containing the nonce.
	nonce := NonceForBody(body)
	doc, err := queryIMDSAttestedDocument(ctx, nonce)
	if err != nil {
		return "", fmt.Errorf("querying attested document: %w", err)
	}
	if doc.Signature == "" {
		return "", fmt.Errorf("empty attested document signature")
	}
	klog.V(2).Infof("Azure authenticator obtained attested document for %q", metadata.ResourceID)

	// Token format: "x-azure-id <resourceID> <base64-pkcs7-signature>"
	return AzureAuthenticationTokenPrefix + metadata.ResourceID + " " + doc.Signature, nil
}

View on GitHub (pinned to 4c8573c808)

Solutions

1. Test the attested endpoint directly: curl -H Metadata:true 'http://169.254.169.254/metadata/attested/document?api-version=2025-04-07&nonce=<hex>' and read the wrapped error/status
2. Retry with backoff on 429/5xx — IMDS throttles aggressively
3. Verify VM/OS image supports attested data and the api-version is current
4. Check for network interference with the link-local IMDS address

// Verify attested document endpoint responds before bootstrap
func attestedOK(nonce string) error {
    c := &http.Client{Timeout: 5 * time.Second, Transport: &http.Transport{Proxy: nil}}
    u := "http://169.254.169.254/metadata/attested/document?api-version=2025-04-07&nonce=" + url.QueryEscape(nonce)
    req, _ := http.NewRequest("GET", u, nil)
    req.Header.Set("Metadata", "true")
    resp, err := c.Do(req)
    if err != nil { return err }
    defer resp.Body.Close()
    if resp.StatusCode != 200 { return fmt.Errorf("attested endpoint status %d", resp.StatusCode) }
    return nil
}

// Back off specifically on 429/503 from the attested endpoint
for attempt := 0; attempt < 5; attempt++ {
    token, err := authenticator.CreateToken(body)
    if err == nil { use(token); break }
    if strings.Contains(err.Error(), "status 429") || strings.Contains(err.Error(), "status 503") {
        time.Sleep(time.Duration(1<<attempt) * time.Second)
        continue
    }
    return err
}

When it happens

Trigger: CreateToken called with a request body; NonceForBody derives a nonce; the inner queryIMDS call fails on request creation, transport error (timeout/refused), non-200 status (e.g. 400/404/429 for attested endpoint), read error, or unmarshal error.

Common situations: IMDS throttling (429) during mass node rollouts; VM image without attested-data support or very old IMDS; API version compatibility issues with the attested endpoint; transient IMDS outage; NSG blocking IMDS.

Related errors

• querying instance metadata: %w
• missing resource ID
• empty attested document signature
• failed to get region from ec2 metadata: %w
• empty subscription ID

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/5561ea47c7c5ce2c. Report an issue: GitHub.