kubernetes/kops · error

request's keypair ID %q for %s didn't match server's %q

Error message

request's keypair ID %q for %s didn't match server's %q

What it means

Since kOps 1.22, bootstrap requests may carry keypairIDs so the server can detect serving a CA that the node does not know about. If the request's keypair ID for the signer differs from the server's current one, issueCert fails rather than issuing a cert against a CA the node would not trust.

Source

Thrown at cmd/kops-controller/pkg/server/server.go:345

		}
		issueReq.AlternateNames = id.CertificateNames
		issueReq.Type = "server"
	case "kube-proxy":
		issueReq.Subject = pkix.Name{
			CommonName: rbac.KubeProxy,
		}
	case "kube-router":
		issueReq.Subject = pkix.Name{
			CommonName: rbac.KubeRouter,
		}
	default:
		return "", fmt.Errorf("unexpected key name")
	}

	// This field was added to the protocol in kOps 1.22.
	if len(keypairIDs) > 0 {
		if keypairIDs[issueReq.Signer] != s.keypairIDs[issueReq.Signer] {
			return "", fmt.Errorf("request's keypair ID %q for %s didn't match server's %q", keypairIDs[issueReq.Signer], issueReq.Signer, s.keypairIDs[issueReq.Signer])
		}
	}

	cert, _, _, err := pki.IssueCert(ctx, issueReq, s.keystore)
	if err != nil {
		return "", fmt.Errorf("issuing certificate: %v", err)
	}

	return cert.AsString()
}

// recovery is responsible for ensuring we don't exit on a panic.
func recovery(next http.Handler) http.Handler {
	return http.HandlerFunc(func(w http.ResponseWriter, req *http.Request) {
		defer func() {
			if err := recover(); err != nil {
				w.WriteHeader(http.StatusInternalServerError)

View on GitHub (pinned to 4c8573c808)

Solutions

1. Re-run bootstrap/nodeup on the node so it fetches fresh keypair IDs and retries
2. Confirm a CA rotation completed and allow nodes to re-bootstrap with updated keypair IDs
3. Check keystore consistency across etcd backups/restores; restore if a partial rotation occurred
4. Upgrade nodes to a kOps version supporting the keypairIDs protocol field

// compare keypair IDs before issuing
for signer, id := range keypairIDs {
    if current, ok := serverKeypairIDs[signer]; ok && id != current {
        return fmt.Errorf("stale keypair ID for %s: have %q want %q", signer, id, current)
    }
}

cert, err := issueCert(ctx, req)
if err != nil && strings.Contains(err.Error(), "didn't match server's") {
    // refresh keypair IDs and re-bootstrap once
    if rerr := refreshKeypairIDsAndRetry(ctx); rerr != nil {
        return rerr
    }
}

When it happens

Trigger: Node bootstraps with cached keypair IDs while the server has rotated the CA / keypair for that signer (e.g. etcd-clients-ca, kubernetes-ca), so the IDs mismatch.

Common situations: CA rotation performed between node bootstrap attempts; stale nodeup state replayed; mixed-version clusters where the node sends keypairIDs from an old keystore.

Related errors

• decoding pem public key
• unexpected key type %q
• parsing key: %v
• key name not enabled
• unexpected key name

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/5f63cfcfab30af4e. Report an issue: GitHub.