kubernetes/kops · error

promoting keypair for %s: %v

Error message

promoting keypair for %s: %v

What it means

This error wraps a per-keyset failure that occurred while running `kops promote keypair all`. RunPromoteKeypair lists every keyset in the cluster's keystore and, for each rotatable keyset, calls promoteKeypair to promote its newest usable keypair. When any individual keyset promotion fails, the error is wrapped as "promoting keypair for <keyset>: <cause>" so the operator knows which keyset aborted the batch.

Source

Thrown at cmd/kops/promote_keypair.go:148

	keyStore, err := clientSet.KeyStore(cluster)
	if err != nil {
		return fmt.Errorf("getting keystore: %v", err)
	}

	if options.Keyset != "all" {
		return promoteKeypair(ctx, out, options.Keyset, options.KeypairID, keyStore)
	}

	keysets, err := keyStore.ListKeysets()
	if err != nil {
		return fmt.Errorf("listing keysets: %v", err)
	}

	for name := range keysets {
		if rotatableKeysetFilter(name, nil) {
			if err := promoteKeypair(ctx, out, name, "", keyStore); err != nil {
				return fmt.Errorf("promoting keypair for %s: %v", name, err)
			}
		}
	}

	return nil
}

func promoteKeypair(ctx context.Context, out io.Writer, name string, keypairID string, keyStore fi.CAStore) error {
	keyset, err := keyStore.FindKeyset(ctx, name)
	if err != nil {
		return fmt.Errorf("reading keyset: %v", err)
	} else if keyset == nil {
		return fmt.Errorf("keyset not found")
	}

	if keypairID == "" {
		highestCandidateId := big.NewInt(0)
		for id, item := range keyset.Items {

View on GitHub (pinned to 4c8573c808)

Solutions

1. Read the wrapped cause after "promoting keypair for <name>: " — it names the keyset and the underlying error.
2. Retry the command once transient state-store/network issues are resolved; already-promoted keysets simply report "no keypair newer than current primary".
3. Promote the failing keyset individually (`kops promote keypair <keyset> ...`) to isolate and debug it.
4. Verify state-store access (`--state` flag / KOPS_STATE_STORE) and that the cluster config is intact with `kops get cluster <name>`.

Example fix

// before: run bulk promotion and get opaque failure
kops promote keypair all --name cluster.example.com
// after: isolate the failing keyset and check state store first
kops get cluster cluster.example.com --state s3://my-state-store
kops promote keypair kubernetes-ca --name cluster.example.com --state s3://my-state-store

// shell pre-check before bulk promotion
kops get cluster "$CLUSTER" --state "$KOPS_STATE_STORE" && \
kops get keypairs --name "$CLUSTER" --state "$KOPS_STATE_STORE" || \
  { echo "state store or keypairs unreadable; aborting promote keypair all"; exit 1; }

if ! kops promote keypair all --name "$CLUSTER" --state "$KOPS_STATE_STORE"; then
  echo "bulk promotion failed; inspect wrapped cause per keyset"
  kops get keypairs --name "$CLUSTER" --state "$KOPS_STATE_STORE"
  # retry individual keysets to find the offender
  for ks in kubernetes-ca kubernetes-front-proxy-ca service-account; do
    kops promote keypair "$ks" --name "$CLUSTER" --state "$KOPS_STATE_STORE" || echo "failed keyset: $ks"
  done
fi

When it happens

Trigger: Running `kops promote keypair all --name <cluster> ...` where at least one rotatable keyset fails inside promoteKeypair — e.g. FindKeyset returns an error (backend/state-store read failure), or the keyset lacks a promotable candidate causing a wrapped inner error. The message shown here is the outer wrapper; the %v carries the real cause (e.g. "reading keyset: ...").

Common situations: State-store (S3/GCS/etc.) connectivity or permission problems during a bulk rotation; a corrupted keyset that fails to read; interrupted rotation leaving a keyset unreadable; running the command against a cluster whose state store was migrated or partially deleted.

Related errors

• keypair has no certificate
• cannot specify --key with "all"
• cannot specify --primary with "all"
• adding keypair to %q is not supported
• error getting keystore: %v

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/fdffe0c67a9c3770. Report an issue: GitHub.