{"record":{"id":"ad7f223b6df5230a","repo":"benbjohnson/litestream","slug":"gs-cannot-delete-object-q-w","errorCode":null,"errorMessage":"gs: cannot delete object %q: %w","messagePattern":"gs: cannot delete object %q: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"gs/replica_client.go","lineNumber":113,"sourceCode":"func (c *ReplicaClient) DeleteAll(ctx context.Context) error {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn err\n\t}\n\n\t// Iterate over every object and delete it.\n\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"LIST\").Inc()\n\tfor it := c.bkt.Objects(ctx, &storage.Query{Prefix: c.Path + \"/\"}); ; {\n\t\tattrs, err := it.Next()\n\t\tif errors.Is(err, iterator.Done) {\n\t\t\tbreak\n\t\t} else if err != nil {\n\t\t\treturn fmt.Errorf(\"failed to list objects in GCS bucket %s (path: %s): %w\", c.Bucket, c.Path, err)\n\t\t}\n\n\t\tif err := c.bkt.Object(attrs.Name).Delete(ctx); isNotExists(err) {\n\t\t\tcontinue\n\t\t} else if err != nil {\n\t\t\treturn fmt.Errorf(\"gs: cannot delete object %q: %w\", attrs.Name, err)\n\t\t}\n\t\tinternal.OperationTotalCounterVec.WithLabelValues(ReplicaClientType, \"DELETE\").Inc()\n\t}\n\n\t// log.Printf(\"%s(%s): retainer: deleting\", r.db.Path(), r.Name())\n\n\treturn nil\n}\n\n// LTXFiles returns an iterator over all available LTX files for a level.\n// GCS always uses accurate timestamps from metadata since they're included in LIST operations at zero cost.\n// The useMetadata parameter is ignored.\nfunc (c *ReplicaClient) LTXFiles(ctx context.Context, level int, seek ltx.TXID, useMetadata bool) (ltx.FileIterator, error) {\n\tif err := c.Init(ctx); err != nil {\n\t\treturn nil, err\n\t}\n\n\tdir := litestream.LTXLevelDir(c.Path, level)","sourceCodeStart":95,"sourceCodeEnd":131,"githubUrl":"https://github.com/benbjohnson/litestream/blob/4ed7a308f6271ebfd2b0a6e4b70b03011a37e4a3/gs/replica_client.go#L95-L131","documentation":"DeleteAll() successfully listed objects but failed to delete a specific object in the bucket. Only genuine delete errors are wrapped here — Object Not Found (isNotExists) is tolerated and skipped. The wrapped cause carries the GCS API error for the named object.","triggerScenarios":"Calling DeleteAll() when the credentials have list but not delete permission (storage.objects.delete denied), the object was concurrently modified/has a retention policy/held, or a transient GCS error occurs on the delete call.","commonSituations":"IAM role grants objectViewer only (can list, can't delete); bucket with retention/bucket-lock policies preventing deletion; object recreated concurrently by another litestream instance; storage class/legal-hold restrictions.","solutions":["Grant storage.objects.delete permission (roles/storage.objectAdmin) to the service account","Check for bucket retention policies or event-based holds locking the objects","Inspect the wrapped error's GCS status code (403 permission vs 412 condition vs 5xx transient)","Verify no second litestream instance is replicating to the same bucket/path concurrently","Retry transient failures"],"exampleFix":"# before: delete denied\ngsutil iam ch serviceAccount:sa@project.iam.gserviceaccount.com:roles/storage.objectAdmin gs://my-bucket\n# also check retention\ngsutil retention ls gs://my-bucket","handlingStrategy":"validation","validationCode":"// preflight delete permission\nobj := bkt.Object(\"__litestream_permcheck__\")\nif err := obj.Delete(ctx); err != nil && !isNotExists(err) {\n    return fmt.Errorf(\"service account lacks delete permission: %w\", err)\n}","typeGuard":null,"tryCatchPattern":"if err := rc.DeleteAll(ctx); err != nil {\n    var ee *googleapi.Error\n    if errors.As(err, &ee) && ee.Code == 403 {\n        // escalate IAM role and retry\n    }\n    return err\n}","preventionTips":["Use a service account with full objectAdmin on the replica bucket","Avoid bucket retention policies/holds on replication buckets","Never run two litestream instances against the same replica path"],"tags":["gcs","cloud-storage","permissions","delete"],"backgroundTag":"permission-denied","analyzedSha":"4ed7a308f6271ebfd2b0a6e4b70b03011a37e4a3","analyzedAt":"2026-09-06T18:29:25.564Z","contentChangedAt":"2026-09-06T18:29:25.564Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}