more than 3 providers (%d) found in secrets encryption

Error message

more than 3 providers (%d) found in secrets encryption

What it means

Before enabling/disabling secrets encryption, k3s parses /var/lib/rancher/k3s/server/cred/encryption-config.yaml into provider blocks. Its rotation state machine only understands configurations with at most three providers (active key, old key(s), identity fallback). More than three providers means the config was produced outside the documented k3s flow and k3s refuses to operate on it.

Source

Thrown at pkg/server/handlers/secrets-encrypt.go:151

		}
	}

	return state, nil
}

func encryptionEnable(ctx context.Context, control *config.Control, enable bool) error {
	providers, err := secretsencrypt.GetEncryptionProviders(control.Runtime)
	// Enable secrets encryption with an identity provider on a cluster that does not have any encryption config
	if err != nil && os.IsNotExist(err) && enable {
		if err := secretsencrypt.WriteIdentityConfig(control); err != nil {
			return err
		}
		return cluster.Save(ctx, control, true)
	} else if err != nil {
		return err
	}
	if len(providers) > 3 {
		return fmt.Errorf("more than 3 providers (%d) found in secrets encryption", len(providers))
	}
	curKeys, err := secretsencrypt.GetEncryptionKeys(control.Runtime)
	if err != nil {
		return err
	}

	if providers[len(providers)-1].Identity != nil && (providers[0].AESCBC != nil || providers[0].Secretbox != nil) && !enable {
		logrus.Infoln("Disabling secrets encryption")
		if err := secretsencrypt.WriteEncryptionConfig(control.Runtime, curKeys, control.EncryptProvider, enable); err != nil {
			return err
		}
	} else if !enable {
		logrus.Infoln("Secrets encryption already disabled")
		return nil
	} else if providers[0].Identity != nil && (providers[1].AESCBC != nil || providers[1].Secretbox != nil) && enable {
		foundKey := false
		// Check the rest of the providers (generally 2nd and 3rd) for the key type we are trying to enable.
		// If we find one, we can proceed.

View on GitHub (pinned to 6ba341e396)

Solutions

Restore an encryption config generated by k3s itself: copy the file from another control-plane node that k3s still manages, or rebuild the config with exactly one active key provider plus identity.
Keep provider count within k3s's supported layout (first-key, second-key, identity) and perform all changes through 'k3s secrets-encrypt' stages instead of editing the file.
Verify with 'k3s secrets-encrypt status' that k3s can parse the repaired config before retrying enable/disable.
Ensure no external tool rewrites the file afterward (exclude it from config management).

Example fix

# before: 4 providers in /var/lib/rancher/k3s/server/cred/encryption-config.yaml
apiVersion: apiserver.config.k8s.io/v1
kind: EncryptionConfiguration
resources:
- providers: [aescbc, aescbc, aescbc, identity]  # -> error

# after: k3s-managed 3-provider layout
- providers: [aescbc(active), identity]  # old key added only during rotate stages

Defensive patterns

Strategy: validation

Validate before calling

// Before enabling, count providers in the config k3s will parse
import "sigs.k8s.io/yaml"

type encConf struct {
    Resources []struct{ Providers []map[string]any `json:"providers"` } `json:"resources"`
}
var c encConf
yaml.Unmarshal(raw, &c)
if n := len(c.Resources[0].Providers); n > 3 {
    log.Fatalf("%d providers - reduce to k3s-managed layout", n)
}

Prevention

Never hand-edit encryption-config.yaml; use k3s secrets-encrypt stages
Exclude the cred directory from config management overwrite
Audit provider count after restores or migrations

When it happens

Trigger: Running 'k3s secrets-encrypt enable/disable' (or POSTing to the secrets-encrypt endpoint) after the encryption config file was hand-edited or externally generated with four or more provider entries, e.g. several manual aescbc/secretbox blocks plus identity.

Common situations: Admins hand-crafting the encryption config for multi-key policies; config management (Ansible/Helm) overwriting the file with a larger provider list; copying a config from a different k8s distribution that does not enforce the 3-provider bound; mixing k3s rotation stages with manual edits.

Related errors

AI-assisted analysis of k3s-io/k3s@6ba341e396 (2026-08-15). Data as JSON: /api/errors/427977351f5f720d. Report an issue: GitHub.