kubernetes/kops · error

error loading certificate %s/%s: %v

Error message

error loading certificate %s/%s: %v

What it means

parseKeyset decodes each keyset item's public material with pki.ParsePEMCertificate. If the stored bytes are not a valid PEM certificate, the item cannot be loaded and the whole keyset parse fails with 'error loading certificate <name>/<id>: <underlying>'. Raised in upup/pkg/fi/clientset_castore.go:89 from loadKeyset and ListKeysets.

Source

Thrown at upup/pkg/fi/clientset_castore.go:89

	name := o.Name

	keyset := &Keyset{
		Items: make(map[string]*KeysetItem),
	}

	for _, key := range o.Spec.Keys {
		ki := &KeysetItem{
			Id: key.Id,
		}
		if key.DistrustTimestamp != nil {
			distrustTimestamp := key.DistrustTimestamp.Time
			ki.DistrustTimestamp = &distrustTimestamp
		}
		if len(key.PublicMaterial) != 0 {
			cert, err := pki.ParsePEMCertificate(key.PublicMaterial)
			if err != nil {
				klog.Warningf("key public material was %s", key.PublicMaterial)
				return nil, fmt.Errorf("error loading certificate %s/%s: %v", name, key.Id, err)
			}
			ki.Certificate = cert
		}

		if len(key.PrivateMaterial) != 0 {
			privateKey, err := pki.ParsePEMPrivateKey(key.PrivateMaterial)
			if err != nil {
				return nil, fmt.Errorf("error loading private key %s/%s: %v", name, key.Id, err)
			}
			ki.PrivateKey = privateKey
		}

		keyset.Items[key.Id] = ki
	}

	keyset.Primary = keyset.Items[FindPrimary(o).Id]

	return keyset, nil

View on GitHub (pinned to 4c8573c808)

Solutions

  1. Inspect key.PublicMaterial (the warning log prints it) and fix or re-encode it as a valid PEM CERTIFICATE block
  2. Re-issue/rotate the keypair for that keyset (e.g. 'kops export kubecfg'/'kops replace' or re-run create keypair) so valid material is stored
  3. Restore the Keyset object from a backup or delete the corrupt item and re-add it

Example fix

// corrupt material
key.PublicMaterial = []byte("not-a-pem")
// after: store valid PEM
key.PublicMaterial = pem.EncodeToMemory(&pem.Block{Type: "CERTIFICATE", Bytes: derBytes})
Defensive patterns

Strategy: try-catch

Validate before calling

for _, b := range [][]byte{item.PublicMaterial} {
	if !pemHasBlock(b, "CERTIFICATE") {
		return fmt.Errorf("keyset item public material is not PEM CERTIFICATE")
	}
}
if _, err := pki.ParsePEMCertificate(item.PublicMaterial); err != nil {
	return fmt.Errorf("certificate will fail to load: %w", err)
}

Type guard

func isPEMCertificate(b []byte) bool {
	block, _ := pem.Decode(b)
	return block != nil && block.Type == "CERTIFICATE"
}

Try / catch

keyset, err := store.FindKeyset(ctx, name)
if err != nil {
	if strings.Contains(err.Error(), "error loading certificate") {
		// quarantine/rotate the corrupted keyset item
	}
	return err
}

Prevention

When it happens

Trigger: A kops Keyset API object whose key.PublicMaterial contains corrupt, truncated, empty-but-nonzero, or non-PEM bytes, so ParsePEMCertificate returns an error.

Common situations: Manual edits to Keyset objects in the cluster registry; data corruption after interrupted writes; exporting/importing keysets across kOps versions that changed the material encoding.

Understand the failure class

Related errors


AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/a1505b004d0e56d1. Report an issue: GitHub.