{"record":{"id":"82ccb982d4ce3700","repo":"benbjohnson/litestream","slug":"abs-cannot-delete-blob-q-w","errorCode":null,"errorMessage":"abs: cannot delete blob %q: %w","messagePattern":"abs: cannot delete blob %q: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"abs/replica_client.go","lineNumber":348,"sourceCode":"\t\tInclude: azblob.ListBlobsInclude{Metadata: true},\n\t})\n\n\tfor pager.More() {\n\t\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"LIST\").Inc()\n\n\t\tresp, err := pager.NextPage(ctx)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"abs: cannot list blobs: %w\", err)\n\t\t}\n\n\t\tfor _, item := range resp.Segment.BlobItems {\n\t\t\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"DELETE\").Inc()\n\n\t\t\t_, err := c.client.DeleteBlob(ctx, c.Bucket, *item.Name, nil)\n\t\t\tif isNotExists(err) {\n\t\t\t\tcontinue\n\t\t\t} else if err != nil {\n\t\t\t\treturn fmt.Errorf(\"abs: cannot delete blob %q: %w\", *item.Name, err)\n\t\t\t}\n\t\t}\n\t}\n\n\treturn nil\n}\n\ntype ltxFileIterator struct {\n\tctx    context.Context\n\tcancel context.CancelFunc\n\tclient *ReplicaClient\n\tlevel  int\n\tseek   ltx.TXID\n\n\tpager     *runtime.Pager[azblob.ListBlobsFlatResponse]\n\tpageItems []*ltx.FileInfo\n\tpageIndex int\n","sourceCodeStart":330,"sourceCodeEnd":366,"githubUrl":"https://github.com/benbjohnson/litestream/blob/4ed7a308f6271ebfd2b0a6e4b70b03011a37e4a3/abs/replica_client.go#L330-L366","documentation":"This error is returned by ReplicaClient.DeleteAll when an individual blob deletion fails during the delete-all sweep. Blobs reported as non-existent are skipped via isNotExists(err), so this indicates a real DELETE failure (permissions, lease, immutability policy, network) for the specific blob named in the message.","triggerScenarios":"DeleteAll(ctx) iterates listed blobs and azblob DeleteBlob fails for one of them with a non-404 error — e.g. blob under a legal hold / immutability policy, credentials lack delete permission, active lease, or transient network failure.","commonSituations":"Deleting a replica whose container has WORM immutability policies; read-only credentials; lease conflicts from concurrent processes; soft-delete plus policy preventing immediate removal.","solutions":["Check the wrapped azblob error for the HTTP status; 403 means missing delete permissions (grant Storage Blob Data Contributor or a SAS with 'd')","Look for immutability policies, legal holds, or active leases on the named blob in the Azure portal and remove them if intended","Retry DeleteAll — it is idempotent; already-deleted or missing blobs are skipped","If the whole replica cannot be purged due to policy, remove the policy scope or recreate the container"],"exampleFix":"// before\nerr := client.DeleteAll(ctx) // blob under legal hold\n// after\nif err := client.DeleteAll(ctx); err != nil {\n\tvar respErr *azcore.ResponseError\n\tif errors.As(err, &respErr) && respErr.ErrorCode == \"BlobIsImmutedWithLegalHold\" {\n\t\t// clear legal hold in Azure portal, then retry\n\t}\n\treturn err\n}","handlingStrategy":"retry","validationCode":"// Go\n// probe delete capability on a scratch blob before sweeping\nprobe := c.client.NewBlockBlobClient(c.Bucket, \"__probe__\", nil)\nif _, err := probe.Upload(ctx, nopReader, nil); err == nil {\n\tif _, err := probe.Delete(ctx, nil); err != nil { return err }\n}","typeGuard":"// Go\nfunc isHoldOrLeaseError(err error) bool {\n\tvar re *azcore.ResponseError\n\tif !errors.As(err, &re) { return false }\n\treturn re.ErrorCode == \"BlobIsImmutedWithLegalHold\" || re.ErrorCode == \"LeaseIdMissing\" || re.StatusCode == http.StatusConflict\n}","tryCatchPattern":"err := client.DeleteAll(ctx)\nif err != nil {\n\tif isHoldOrLeaseError(err) { /* clear hold/lease in portal, then retry */ }\n\telse if transient(err) { err = retryWithBackoff(func() error { return client.DeleteAll(ctx) }) }\n}\nreturn err","preventionTips":["Do not enable WORM immutability/legal holds on containers Litestream must purge","Use credentials with delete rights for replica cleanup tooling","Re-run DeleteAll on failure; it skips blobs already gone","Log per-blob error codes to pinpoint policy-blocked blobs"],"tags":["azure","storage","delete","permissions","network"],"backgroundTag":"blob-delete-failed","analyzedSha":"4ed7a308f6271ebfd2b0a6e4b70b03011a37e4a3","analyzedAt":"2026-09-06T18:29:25.564Z","contentChangedAt":"2026-09-06T18:29:25.564Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}