kubernetes/kops · critical

key %q not found

Error message

key %q not found

What it means

loadCertificates populates the nodeup config's CA trust store (config.CAs) from a map of fi.Keyset objects built by BuildConfig. This error is thrown when the keyset map has no entry for the requested key name, meaning a required keypair (e.g. kubernetes-ca, etcd-clients-ca) was never loaded or does not exist in the cluster's key store.

Source

Thrown at pkg/nodemodel/nodeupconfigbuilder.go:466

		configServer.TLSServerName = kopsControllerName
	}

	for _, host := range hosts {
		baseURL := url.URL{
			Scheme: "https",
			Host:   net.JoinHostPort(host, strconv.Itoa(wellknownports.KopsControllerPort)),
			Path:   "/",
		}
		configServer.Servers = append(configServer.Servers, baseURL.String())
	}

	return configServer
}

func loadCertificates(keysets map[string]*fi.Keyset, name string, config *nodeup.Config, includeKeypairID bool) error {
	keyset := keysets[name]
	if keyset == nil {
		return fmt.Errorf("key %q not found", name)
	}
	certificates, err := keyset.ToCertificateBytes()
	if err != nil {
		return fmt.Errorf("failed to read %q certificates: %w", name, err)
	}
	config.CAs[name] = string(certificates)
	if includeKeypairID {
		if keyset.Primary == nil || keyset.Primary.Id == "" {
			return fmt.Errorf("key %q did not have primary id set", name)
		}
		config.KeypairIDs[name] = keyset.Primary.Id
	}
	return nil
}

// buildWarmPoolImages returns a list of container images that should be pre-pulled during instance pre-initialization
func (n *nodeUpConfigBuilder) buildWarmPoolImages(ig *kops.InstanceGroup) []string {
	if ig == nil || ig.Spec.Role.HasControlPlane() {

View on GitHub (pinned to 4c8573c808)

Solutions

1. Run `kops update cluster <name> --yes` (or `kops create`) so kOps regenerates the missing keyset in the key store, then retry.
2. List existing keypairs with `kops get secrets` / the keypair API to confirm which key is missing; check the quoted name in the error.
3. If keypairs were deleted, restore from the state store backup or rotate: `kops toolbox pki` / keypair rotation docs to recreate the CA.
4. Verify the state store (KOPS_STATE_STORE) points at the right cluster so the builder reads the correct key store.

Example fix

// before: nodeup config built without running cluster update
$ kops replace -f cluster.yaml && kops create secrets never run
// after
$ kops update cluster mycluster.k8s.local --yes   # creates missing kubernetes-ca et al.

names := []string{"kubernetes-ca", "etcd-clients-ca", "etcd-peers-ca", ...}
for _, name := range names {
    if keysets[name] == nil {
        return fmt.Errorf("required keyset %q missing; run kops update cluster to create it", name)
    }
}

func keysetExists(keysets map[string]*fi.Keyset, name string) bool {
    ks, ok := keysets[name]
    return ok && ks != nil
}

if err := loadCertificates(keysets, "kubernetes-ca", config, true); err != nil {
    if strings.Contains(err.Error(), "not found") {
        // regenerate missing keysets via kops update cluster before retrying
        return fmt.Errorf("cluster keystore incomplete (%w); run 'kops update cluster --yes'", err)
    }
    return err
}

When it happens

Trigger: BuildConfig calls loadCertificates for each required key name (nodeup config for a node/control-plane instance group); if a key such as "kubernetes-ca" or "etcd-clients-ca" is missing from the keyset map — because it was never created, was deleted from the key store, or the builder skipped loading it for that role — the error fires.

Common situations: Cluster keypair store damaged or partially deleted (kops get secrets shows missing keys); building nodeup config for a control-plane node in a cluster whose etcd CA keysets were never provisioned; upgrading kOps versions where new keysets (e.g. service-account) must exist but `kops update cluster` was never run to create them; pointing kOps at a cluster with an incomplete state store.

Related errors

• failed to read %q certificates: %w
• key %q did not have primary id set
• missing keypairID for signer %s
• no keypair with id %s for %s
• error generating private key: %v

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/35ed6adab501e73e. Report an issue: GitHub.