{"record":{"id":"483aab4bb29f9b99","repo":"kubernetes/kops","slug":"removing-file-s-w","errorCode":null,"errorMessage":"removing file %s: %w","messagePattern":"removing file (.+?): %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"util/pkg/vfs/azureblob.go","lineNumber":246,"sourceCode":"\tif err != nil {\n\t\treturn err\n\t}\n\n\treturn nil\n}\n\nfunc (p *AzureBlobPath) RemoveAll(ctx context.Context) error {\n\tklog.V(8).Infof(\"Removing ALL files: %s\", p)\n\n\ttree, err := p.ReadTree(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfor _, blobPath := range tree {\n\t\terr := blobPath.Remove(ctx)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"removing file %s: %w\", blobPath, err)\n\t\t}\n\t}\n\n\treturn nil\n}\n\nfunc (p *AzureBlobPath) RemoveAllVersions(ctx context.Context) error {\n\tklog.V(8).Infof(\"Removing ALL file versions: %s\", p)\n\n\ttree, err := p.ReadTree(ctx)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tfor _, blobPath := range tree {\n\t\terr := blobPath.Remove(ctx)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"removing file %s: %w\", blobPath, err)","sourceCodeStart":228,"sourceCodeEnd":264,"githubUrl":"https://github.com/kubernetes/kops/blob/4c8573c808a73d578c5eadc86d410646ea0b0d73/util/pkg/vfs/azureblob.go#L228-L264","documentation":"AzureBlobPath.RemoveAll lists all blobs under the path (the tree) and deletes each individually; if any single blob Remove fails, the operation aborts and wraps the error as `removing file %s: %w` with the blob path and underlying cause (403, 404 handling, lease conflicts, network).","triggerScenarios":"Calling RemoveAll on an azureblob VFSPath where one or more blobs under the prefix cannot be deleted — e.g. missing storage-account key or SAS lacking delete permission, blob immutable/leased, container missing, or transient API failure mid-deletion.","commonSituations":"`kops delete cluster` against an Azure state store with a read-only or deletion-restricted SAS token; soft-delete/immutability policies on the storage account blocking blob deletion; expired SAS token partway through a long tree delete.","solutions":["Regenerate credentials/SAS with delete (and list) permissions, e.g. account key or SAS with d+l flags.","Check for Azure soft-delete / immutability policies or active leases on the blobs and disable/allow delete.","Retry the operation — RemoveAll is resumable since already-deleted blobs are simply skipped on the next listing.","Verify the container exists and the account name/key in your Azure config are correct."],"exampleFix":"// before: SAS without delete\nsas := \"?sv=...&sp=rl\" // read+list only\n// after\nsas := \"?sv=...&sp=rl d\" // sp=rld : add delete permission","handlingStrategy":"retry","validationCode":"// Go: verify delete permission before RemoveAll\nsvc, _ := azblob.NewClient(\"https://acct.blob.core.windows.net/\", cred)\n_, err := svc.DeleteBlob(ctx, \"container\", \"probe-blob\", nil)\nif err != nil {\n    if azerr, ok := err.(*azcore.ResponseError); ok && azerr.StatusCode == 403 {\n        return fmt.Errorf(\"credentials lack delete permission on container\")\n    }\n}\n// probe blob may not exist; 404 is fine — 403 means perms problem","typeGuard":"func isAuthOrPolicyError(err error) bool {\n    var re *azcore.ResponseError\n    if errors.As(err, &re) {\n        return re.StatusCode == 403 || re.StatusCode == 409\n    }\n    return false\n}","tryCatchPattern":"err := path.RemoveAll(ctx)\nif err != nil {\n    if isAuthOrPolicyError(err) {\n        // fix SAS/IAM or immutability policy, then retry\n    }\n    // transient: retry with backoff; RemoveAll resumes since deleted blobs vanish from listing\n    retry.Do(func() error { return path.RemoveAll(ctx) }, retry.Attempts(3))\n}","preventionTips":["Use account keys or SAS tokens including delete (d) and list (l) permissions for delete operations","Disable or account for soft-delete/immutability policies before mass deletes","Ensure SAS tokens outlive long-running delete operations","Delete by retrying: tree delete is resumable and idempotent"],"tags":["azure","blob-storage","deletion","permissions"],"backgroundTag":"blob-delete-failed","analyzedSha":"4c8573c808a73d578c5eadc86d410646ea0b0d73","analyzedAt":"2026-09-05T04:13:19.212Z","contentChangedAt":"2026-09-05T04:13:19.212Z","schemaVersion":2},"datasetVersion":"2026-09-12T07:17:12.445Z"}