kubernetes/kops · error

error removing cluster from state store: %v

Error message

error removing cluster from state store: %v

What it means

After deleting (or while finalizing) cluster deletion, RunDeleteCluster calls clientset.DeleteCluster to remove the cluster's configuration from the kOps state store (S3/GCS/etc.). If the registry delete fails, the underlying error is wrapped with this message.

Source

Thrown at cmd/kops/delete_cluster.go:222

		}
	}

	if !options.External {
		if !options.Yes {
			if wouldDeleteCloudResources {
				fmt.Fprintf(out, "\nMust specify --yes to delete cloud resources & unregister cluster\n")
			} else {
				fmt.Fprintf(out, "\nMust specify --yes to unregister the cluster\n")
			}
			return nil
		}
		clientset, err := f.KopsClient()
		if err != nil {
			return err
		}
		err = clientset.DeleteCluster(ctx, cluster)
		if err != nil {
			return fmt.Errorf("error removing cluster from state store: %v", err)
		}
	}

	b := kubeconfig.NewKubeconfigBuilder()
	b.Context = clusterName
	err = b.DeleteKubeConfig(clientcmd.NewDefaultPathOptions())
	if err != nil {
		klog.Warningf("error removing kube config: %v", err)
	}

	fmt.Fprintf(out, "\nDeleted cluster: %q\n", clusterName)
	return nil
}

func completeRegion(cmd *cobra.Command, args []string, toComplete string) ([]string, cobra.ShellCompDirective) {
	// TODO call into cloud provider(s) to get list of valid regions
	return nil, cobra.ShellCompDirectiveNoFileComp
}

View on GitHub (pinned to 4c8573c808)

Solutions

  1. Read the wrapped %v error to identify the backend failure and fix permissions (e.g. grant s3:DeleteObject on the state store prefix).
  2. Verify the cluster config still exists: `kops get cluster --name <name>`; if already gone, the delete can be considered done.
  3. Retry after network issues; if the state store entry is stuck, remove it manually with `kops delete cluster` again or delete the S3 objects under /clusters/<name>.

Example fix

// before (IAM policy)
{"Action":["s3:ListBucket","s3:GetObject"], ...}
// after
{"Action":["s3:ListBucket","s3:GetObject","s3:DeleteObject","s3:DeleteObjectVersion"], ...}
Defensive patterns

Strategy: retry

Validate before calling

if err := exec.Command("kops", "get", "cluster", "--name", clusterName).Run(); err != nil {
    // cluster config already absent from state store; skip delete
}

Try / catch

if err := runDeleteCluster(...); err != nil && strings.Contains(err.Error(), "state store") {
    // inspect wrapped cause; fix IAM/network; retry with backoff
}

Prevention

When it happens

Trigger: State store backend errors during DeleteCluster: insufficient IAM permissions on the S3 bucket, object already removed/concurrency conflict, network failure to the state store, or a non-empty/locked registry.

Common situations: S3 bucket policy denies s3:DeleteObject; state store was manually cleaned so the cluster config no longer exists; transient network outage; credentials scoped to read-only.

Related errors


AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/207e2ccbf08fe1f2. Report an issue: GitHub.