k3s-io/k3s · critical
bootstrap data already found and encrypted with different to
Error message
bootstrap data already found and encrypted with different token
What it means
In getBootstrapKeyFromStorage, bootstrap data exists under '/bootstrap' but no stored key matches tokenKey (the hash of the currently supplied normalized token), and the migration candidates (empty-string key, old-token key) also did not match. The bootstrap is therefore encrypted with a different token than the one presented.
Source
Thrown at pkg/cluster/storage.go:341
}
} else {
logrus.Debugf("Skipping bootstrap token migration checks: storage client %T is not writable", storageClient)
}
// getting the list of bootstrap again after migrating the empty key
bootstrapList, err = storageClient.List(ctx, "/bootstrap", 0)
if err != nil {
return nil, false, err
}
for _, bootstrapKV := range bootstrapList {
// ensure bootstrap is stored in the current token's key
logrus.Debugf("checking bootstrap key %s against %s", string(bootstrapKV.Key), tokenKey)
if string(bootstrapKV.Key) == tokenKey {
return &bootstrapKV, false, nil
}
}
return nil, false, errors.New("bootstrap data already found and encrypted with different token")
}
// migrateTokens will list all keys that has prefix /bootstrap and will check for key that is
// hashed with empty string and keys that is hashed with old token format before normalizing
// then migrate those and resave only with the normalized token
func migrateTokens(ctx context.Context, bootstrapList []mvccpb.KeyValue, storageClient store.ReadWriteCloser, emptyStringKey, tokenKey, token, oldToken string) error {
oldTokenKey := storageKey(oldToken)
for _, bootstrapKV := range bootstrapList {
// checking for empty string bootstrap key
logrus.Debug("Comparing ", string(bootstrapKV.Key), " to ", oldTokenKey)
if string(bootstrapKV.Key) == emptyStringKey {
logrus.Warn("Bootstrap data encrypted with empty string, deleting and resaving with token")
if err := doMigrateToken(ctx, storageClient, bootstrapKV, "", emptyStringKey, token, tokenKey); err != nil {
return err
}
} else if string(bootstrapKV.Key) == oldTokenKey && oldTokenKey != tokenKey {
if emptyStringKey != "" {View on GitHub (pinned to 6ba341e396)
Solutions
- Restore and use the original full token (K10<CA hash>::server:<secret>) that the first server generated; it is the passphrase for the stored bootstrap.
- If the original token is lost but a healthy server still runs, read the current valid token from that server's /var/lib/rancher/k3s/server/token.
- As a last resort, reinitialize the cluster from snapshots with a new token.
Defensive patterns
Strategy: validation
Validate before calling
// Verify the presented token maps to an existing key before starting:
norm, _ := util.NormalizeToken(token)
want := storageKey(norm) // "/bootstrap/" + ShortHash(norm, 12)
list, _ := storageClient.List(ctx, "/bootstrap", 0)
found := false
for _, kv := range list { if string(kv.Key) == want { found = true } }
if !found { return errors.New("token does not match stored bootstrap key: use the original cluster token") } Try / catch
if err := startServer(); err != nil {
if strings.Contains(err.Error(), "encrypted with different token") {
// stop: no code fix; supply the original token or reinit from snapshot
}
} Prevention
- Store the original K10...::server:... token in a secret manager at cluster creation and always source it from there.
- Copy the full token including both halves; truncation silently changes the hash.
- If you must rotate tokens, follow the documented migration flow, not a flag change.
When it happens
Trigger: Starting/joining with a token different from the one that encrypted the stored bootstrap; token file and CLI token disagree; partial token rotation where only some servers were updated.
Common situations: K3S_TOKEN changed after initial cluster creation; node reinstalled and rejoined with a new agent/server token of the wrong value; copy-paste of only the secret half of the token.
Related errors
- no bootstrap data found in datastore - check server token va
- no bootstrap data is available to reconcile against
- invalid cipher text, not : delimited
- found multiple bootstrap keys in storage
- --token is required
AI-assisted analysis of k3s-io/k3s@6ba341e396 (2026-08-15).
Data as JSON: /api/errors/5af9037e86896f4a.
Report an issue: GitHub.