{"record":{"id":"677cf49a5342837c","repo":"kubernetes/kops","slug":"waiting-for-kubernetes-api-to-be-served-w","errorCode":null,"errorMessage":"waiting for kubernetes API to be served: %w","messagePattern":"waiting for kubernetes API to be served: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"cmd/kops/reconcile_cluster.go","lineNumber":172,"sourceCode":"\t{\n\t\topt := &ValidateClusterOptions{}\n\t\topt.InitDefaults()\n\t\topt.ClusterName = c.ClusterName\n\t\topt.CreateKubecfgOptions = options.CreateKubecfgOptions\n\t\topt.wait = 10 * time.Minute\n\n\t\t// filter the instance group to only include the control plane\n\t\topt.filterInstanceGroups = func(ig *kops.InstanceGroup) bool {\n\t\t\treturn ig.Spec.Role.HasAPIServer() || ig.Spec.Role.HasControlPlane()\n\t\t}\n\n\t\t// Ignore all pods, we just want to check the control plane is responding\n\t\topt.filterPodsForValidation = func(pod *v1.Pod) bool {\n\t\t\treturn false\n\t\t}\n\n\t\tif _, err := RunValidateCluster(ctx, f, out, opt); err != nil {\n\t\t\treturn fmt.Errorf(\"waiting for kubernetes API to be served: %w\", err)\n\t\t}\n\t}\n\n\tfmt.Fprintf(out, \"Performing rolling-update for control plane\\n\")\n\t{\n\t\topt := &RollingUpdateOptions{}\n\t\topt.InitDefaults()\n\t\topt.ClusterName = c.ClusterName\n\t\topt.CreateKubecfgOptions = options.CreateKubecfgOptions\n\t\topt.InstanceGroupRoles = []string{\n\t\t\tstring(kops.InstanceGroupRoleAPIServer),\n\t\t\tstring(kops.InstanceGroupRoleControlPlane),\n\t\t}\n\t\topt.Yes = c.Yes\n\t\tif err := RunRollingUpdateCluster(ctx, f, out, opt); err != nil {\n\t\t\treturn err\n\t\t}\n\t}","sourceCodeStart":154,"sourceCodeEnd":190,"githubUrl":"https://github.com/kubernetes/kops/blob/4c8573c808a73d578c5eadc86d410646ea0b0d73/cmd/kops/reconcile_cluster.go#L154-L190","documentation":"After updating the control plane configuration, RunReconcileCluster validates that the Kubernetes API is being served (via RunValidateCluster with a 10-minute wait, only control-plane instance groups, pods ignored) before starting the control-plane rolling update. If that validation fails within the wait window, the underlying validation error is wrapped as \"waiting for kubernetes API to be served: %w\". This means the API server is not reachable/healthy after the update step, so reconcile cannot safely proceed to rolling updates.","triggerScenarios":"RunValidateCluster returns an error during the post-update wait: API server pods crash-looping or not yet started after an update, unreachable API endpoint (DNS/LB not provisioned or misconfigured), kubeconfig/credentials cannot authenticate to the cluster, network/firewall blocking the API port, or validation timing out after 10 minutes on a slow control plane.","commonSituations":"Upgrading a cluster where the API server fails to come up (bad manifest, kube-apiserver image pull failure); new clusters where the load balancer or DNS record isn't ready yet; VPN/firewall rules preventing the local machine from reaching the API endpoint; expired or missing admin credentials in kubeconfig; heavily loaded or single-node control planes exceeding the 10-minute validation wait.","solutions":["Read the wrapped error for the root cause; run `kops validate cluster <name>` manually to see current API health and which instance groups fail.","Check control-plane health: SSH/kubectl to the API server nodes and inspect kube-apiserver pods/logs (`kubectl -n kube-system get pods`, or via the node directly) for crash loops.","Verify network path to the API: DNS resolution of the cluster name, load balancer health, security groups/firewall allowing TCP 443, and valid kubeconfig credentials (`kops export kubecfg <name> --admin`).","If the API is merely slow to start, re-run `kops reconcile cluster --yes` once the API responds — validate-then-roll is idempotent and will resume at the same step.","For persistent failures, fix the underlying update issue (e.g. re-run update with corrections, check kOps version compatibility) before retrying reconcile."],"exampleFix":"// before: reconcile fails while waiting for API\n// kops reconcile cluster k8s.example.com --yes\n// -> waiting for kubernetes API to be served: cluster \"k8s.example.com\" has no running instance groups...\n// after: restore API access, then validate and retry\n// kops export kubecfg k8s.example.com --admin\n// kops validate cluster k8s.example.com\n// kops reconcile cluster k8s.example.com --yes","handlingStrategy":"retry","validationCode":"// Before reconcile, confirm the API is already healthy:\n// kops validate cluster <name>\n// or programmatically: run RunValidateCluster and only proceed on success","typeGuard":null,"tryCatchPattern":"if _, err := RunValidateCluster(ctx, f, out, opt); err != nil {\n    // retry with backoff up to the wait budget before failing reconcile\n    return retry.Do(\n        func() error { return fmt.Errorf(\"waiting for kubernetes API to be served: %w\", err) },\n        retry.Attempts(5), retry.Delay(30*time.Second),\n    )\n}","preventionTips":["Run `kops validate cluster` before reconcile so API connectivity/credentials issues surface early.","Ensure kubeconfig is fresh (`kops export kubecfg <name> --admin`) and the API endpoint is reachable (DNS, LB, firewalls).","Verify control-plane instance group updates succeeded (check kube-apiserver pod status) before launching reconcile in CI.","Budget extra time for slow/large control planes; if the 10-minute validation window is regularly exceeded, fix underlying slowness first, then re-run reconcile — it resumes safely.","Treat this as transient where possible: re-running reconcile after the API is up continues from the validation step."],"tags":["api-server","validation","reconcile","cluster-health","timeout"],"backgroundTag":"kubernetes-api-unreachable","analyzedSha":"4c8573c808a73d578c5eadc86d410646ea0b0d73","analyzedAt":"2026-09-05T04:13:19.212Z","contentChangedAt":"2026-09-05T04:13:19.212Z","schemaVersion":2},"datasetVersion":"2026-09-12T07:17:12.445Z"}