benbjohnson/litestream · error

abs: cannot delete blob %q: %w

Error message

abs: cannot delete blob %q: %w

What it means

This error is returned by ReplicaClient.DeleteAll when an individual blob deletion fails during the delete-all sweep. Blobs reported as non-existent are skipped via isNotExists(err), so this indicates a real DELETE failure (permissions, lease, immutability policy, network) for the specific blob named in the message.

Source

Thrown at abs/replica_client.go:348

		Include: azblob.ListBlobsInclude{Metadata: true},
	})

	for pager.More() {
		internal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, "LIST").Inc()

		resp, err := pager.NextPage(ctx)
		if err != nil {
			return fmt.Errorf("abs: cannot list blobs: %w", err)
		}

		for _, item := range resp.Segment.BlobItems {
			internal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, "DELETE").Inc()

			_, err := c.client.DeleteBlob(ctx, c.Bucket, *item.Name, nil)
			if isNotExists(err) {
				continue
			} else if err != nil {
				return fmt.Errorf("abs: cannot delete blob %q: %w", *item.Name, err)
			}
		}
	}

	return nil
}

type ltxFileIterator struct {
	ctx    context.Context
	cancel context.CancelFunc
	client *ReplicaClient
	level  int
	seek   ltx.TXID

	pager     *runtime.Pager[azblob.ListBlobsFlatResponse]
	pageItems []*ltx.FileInfo
	pageIndex int

View on GitHub (pinned to 4ed7a308f6)

Solutions

1. Check the wrapped azblob error for the HTTP status; 403 means missing delete permissions (grant Storage Blob Data Contributor or a SAS with 'd')
2. Look for immutability policies, legal holds, or active leases on the named blob in the Azure portal and remove them if intended
3. Retry DeleteAll — it is idempotent; already-deleted or missing blobs are skipped
4. If the whole replica cannot be purged due to policy, remove the policy scope or recreate the container

Example fix

// before
err := client.DeleteAll(ctx) // blob under legal hold
// after
if err := client.DeleteAll(ctx); err != nil {
	var respErr *azcore.ResponseError
	if errors.As(err, &respErr) && respErr.ErrorCode == "BlobIsImmutedWithLegalHold" {
		// clear legal hold in Azure portal, then retry
	}
	return err
}

// Go
// probe delete capability on a scratch blob before sweeping
probe := c.client.NewBlockBlobClient(c.Bucket, "__probe__", nil)
if _, err := probe.Upload(ctx, nopReader, nil); err == nil {
	if _, err := probe.Delete(ctx, nil); err != nil { return err }
}

// Go
func isHoldOrLeaseError(err error) bool {
	var re *azcore.ResponseError
	if !errors.As(err, &re) { return false }
	return re.ErrorCode == "BlobIsImmutedWithLegalHold" || re.ErrorCode == "LeaseIdMissing" || re.StatusCode == http.StatusConflict
}

err := client.DeleteAll(ctx)
if err != nil {
	if isHoldOrLeaseError(err) { /* clear hold/lease in portal, then retry */ }
	else if transient(err) { err = retryWithBackoff(func() error { return client.DeleteAll(ctx) }) }
}
return err

When it happens

Trigger: DeleteAll(ctx) iterates listed blobs and azblob DeleteBlob fails for one of them with a non-404 error — e.g. blob under a legal hold / immutability policy, credentials lack delete permission, active lease, or transient network failure.

Common situations: Deleting a replica whose container has WORM immutability policies; read-only credentials; lease conflicts from concurrent processes; soft-delete plus policy preventing immediate removal.

Related errors

• abs: cannot delete ltx file %q: %w
• abs: cannot list blobs: %w
• oss: delete batch of %d objects: %w
• oss: failed to delete files:
• close iterator: %w

AI-assisted analysis of benbjohnson/litestream@4ed7a308f6 (2026-09-06). Data as JSON: /api/errors/82ccb982d4ce3700. Report an issue: GitHub.