{"record":{"id":"81df433222ac45e0","repo":"kubernetes/kops","slug":"waiting-for-vmss-deletion-completion-w","errorCode":null,"errorMessage":"waiting for VMSS deletion completion: %w","messagePattern":"waiting for VMSS deletion completion: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"upup/pkg/fi/cloudup/azure/vmscaleset.go","lineNumber":94,"sourceCode":"\nfunc (c *vmScaleSetsClientImpl) Get(ctx context.Context, resourceGroupName string, vmssName string) (*compute.VirtualMachineScaleSet, error) {\n\topts := &compute.VirtualMachineScaleSetsClientGetOptions{\n\t\tExpand: to.Ptr(compute.ExpandTypesForGetVMScaleSetsUserData),\n\t}\n\tresp, err := c.c.Get(ctx, resourceGroupName, vmssName, opts)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"getting VMSS: %w\", err)\n\t}\n\treturn &resp.VirtualMachineScaleSet, nil\n}\n\nfunc (c *vmScaleSetsClientImpl) Delete(ctx context.Context, resourceGroupName, vmssName string) error {\n\tfuture, err := c.c.BeginDelete(ctx, resourceGroupName, vmssName, nil)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"deleting VMSS: %w\", err)\n\t}\n\tif _, err := future.PollUntilDone(ctx, nil); err != nil {\n\t\treturn fmt.Errorf(\"waiting for VMSS deletion completion: %w\", err)\n\t}\n\treturn nil\n}\n\nfunc newVMScaleSetsClientImpl(subscriptionID string, cred *azidentity.DefaultAzureCredential) (*vmScaleSetsClientImpl, error) {\n\tc, err := compute.NewVirtualMachineScaleSetsClient(subscriptionID, cred, nil)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"creating VMSSs client: %w\", err)\n\t}\n\treturn &vmScaleSetsClientImpl{\n\t\tc: c,\n\t}, nil\n}\n","sourceCodeStart":76,"sourceCodeEnd":108,"githubUrl":"https://github.com/kubernetes/kops/blob/4c8573c808a73d578c5eadc86d410646ea0b0d73/upup/pkg/fi/cloudup/azure/vmscaleset.go#L76-L108","documentation":"This error wraps a failure reported by the ARM async long-running-operation poller (future.PollUntilDone) after the delete request was accepted. The VMSS deletion either failed inside Azure (e.g. dependent resources, provisioning error) or polling itself hit timeouts/HTTP errors. The wrapped error carries the poller's terminal state or the intermediate HTTP failure.","triggerScenarios":"BeginDelete returned a future successfully, then PollUntilDone observes: the LRO reached a Failed state (Azure could not delete underlying NICs/disks/VM instances), HTTP polling errors (401 token expiry mid-operation, 429 throttling of the polling GET), or the ctx passed to PollUntilDone was cancelled/timed out before completion.","commonSituations":"Long VMSS deletions exceeding the caller's context timeout (kops delete cluster run under a CI timeout); Azure region outage or capacity issues blocking instance deletion; deletion blocked by resources locked (ReadOnly/Delete locks) in the portal; identity token expiring during very long polls.","solutions":["Increase the context deadline passed to Delete (long LROs can take 10-30+ min for large scale sets)","Check for Azure resource locks (az lock list) on the resource group or scale set and remove conflicting locks","Inspect the failed LRO details: unwrap and read the ResponseError to find the ARM error code for the underlying instance delete failure","Retry the whole Delete; a partially-deleted VMSS can be re-deleted idempotently","Verify no cancellation source (kops abort/CI timeout) is killing ctx mid-poll"],"exampleFix":"// before\nctx, cancel := context.WithTimeout(context.Background(), 5*time.Minute)\ndefer cancel()\nerr := client.Delete(ctx, rg, vmssName)\n// after\nctx, cancel := context.WithTimeout(context.Background(), 30*time.Minute) // LRO can be slow\ndefer cancel()\nerr := client.Delete(ctx, rg, vmssName)","handlingStrategy":"retry","validationCode":"// Go: use a generous deadline for the long-running delete and check locks first\nctx, cancel := context.WithTimeout(context.Background(), 30*time.Minute)\ndefer cancel()\n// pre-check for Azure locks that would fail the LRO:\n// locks := locksClient.ListByResourceGroup(ctx, rg, nil) // abort if a Delete lock exists","typeGuard":"func isLRORetries(err error) bool {\n\tvar respErr *azcore.ResponseError\n\treturn errors.As(err, &respErr) && (respErr.StatusCode == 429 || respErr.StatusCode >= 500 || errors.Is(err, context.DeadlineExceeded))\n}","tryCatchPattern":"err := client.Delete(ctx, rg, vmssName)\nif err != nil {\n\tif isLRORetries(err) {\n\t\t// re-issue Delete with backoff; LROs are safely re-runnable\n\t\treturn retryWithBackoff(ctx, func() error { return client.Delete(ctx, rg, vmssName) })\n\t}\n\treturn fmt.Errorf(\"waiting for VMSS deletion completion: %w\", err)\n}","preventionTips":["Always give VMSS delete LROs a 20-30 minute context, not a default short one","Remove Azure resource locks before cluster teardown","Handle ctx.Canceled/DeadlineExceeded explicitly so users see a clear timeout message","Retry the whole delete idempotently after transient poll failures","Check respErr details for the underlying failed resource when the LRO terminal state is Failed"],"tags":["azure","vmss","delete","long-running-operation","timeout"],"backgroundTag":"long-running-operation-failed","analyzedSha":"4c8573c808a73d578c5eadc86d410646ea0b0d73","analyzedAt":"2026-09-05T04:13:19.212Z","contentChangedAt":"2026-09-05T04:13:19.212Z","schemaVersion":2},"datasetVersion":"2026-09-12T07:17:12.445Z"}