kubernetes/kops · error
error removing cluster from state store: %v
Error message
error removing cluster from state store: %v
What it means
After deleting (or while finalizing) cluster deletion, RunDeleteCluster calls clientset.DeleteCluster to remove the cluster's configuration from the kOps state store (S3/GCS/etc.). If the registry delete fails, the underlying error is wrapped with this message.
Source
Thrown at cmd/kops/delete_cluster.go:222
}
}
if !options.External {
if !options.Yes {
if wouldDeleteCloudResources {
fmt.Fprintf(out, "\nMust specify --yes to delete cloud resources & unregister cluster\n")
} else {
fmt.Fprintf(out, "\nMust specify --yes to unregister the cluster\n")
}
return nil
}
clientset, err := f.KopsClient()
if err != nil {
return err
}
err = clientset.DeleteCluster(ctx, cluster)
if err != nil {
return fmt.Errorf("error removing cluster from state store: %v", err)
}
}
b := kubeconfig.NewKubeconfigBuilder()
b.Context = clusterName
err = b.DeleteKubeConfig(clientcmd.NewDefaultPathOptions())
if err != nil {
klog.Warningf("error removing kube config: %v", err)
}
fmt.Fprintf(out, "\nDeleted cluster: %q\n", clusterName)
return nil
}
func completeRegion(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) {
// TODO call into cloud provider(s) to get list of valid regions
return nil, cobra.ShellCompDirectiveNoFileComp
}View on GitHub (pinned to 4c8573c808)
Solutions
- Read the wrapped %v error to identify the backend failure and fix permissions (e.g. grant s3:DeleteObject on the state store prefix).
- Verify the cluster config still exists: `kops get cluster --name <name>`; if already gone, the delete can be considered done.
- Retry after network issues; if the state store entry is stuck, remove it manually with `kops delete cluster` again or delete the S3 objects under /clusters/<name>.
Example fix
// before (IAM policy)
{"Action":["s3:ListBucket","s3:GetObject"], ...}
// after
{"Action":["s3:ListBucket","s3:GetObject","s3:DeleteObject","s3:DeleteObjectVersion"], ...} Defensive patterns
Strategy: retry
Validate before calling
if err := exec.Command("kops", "get", "cluster", "--name", clusterName).Run(); err != nil {
// cluster config already absent from state store; skip delete
} Try / catch
if err := runDeleteCluster(...); err != nil && strings.Contains(err.Error(), "state store") {
// inspect wrapped cause; fix IAM/network; retry with backoff
} Prevention
- Grant s3:DeleteObject/DeleteObjectVersion on the state store prefix.
- Verify KOPS_STATE_STORE connectivity before deletion.
- Treat 'already gone' from the wrapped error as success in idempotent scripts.
When it happens
Trigger: State store backend errors during DeleteCluster: insufficient IAM permissions on the S3 bucket, object already removed/concurrency conflict, network failure to the state store, or a non-empty/locked registry.
Common situations: S3 bucket policy denies s3:DeleteObject; state store was manually cleaned so the cluster config no longer exists; transient network outage; credentials scoped to read-only.
Related errors
- error loading NodeupConfig %q: %v
- writing keyset: %v
- checking if bucket was public: %w
- error reading last kops version used to update: %v
- error reading directory %q: %v
AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05).
Data as JSON: /api/errors/207e2ccbf08fe1f2.
Report an issue: GitHub.