{"record":{"id":"36d4703180cf9ed6","repo":"benbjohnson/litestream","slug":"failed-to-delete-files","errorCode":null,"errorMessage":"failed to delete files:","messagePattern":"failed to delete files:","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"s3/replica_client.go","lineNumber":1859,"sourceCode":"\tvar apiErr smithy.APIError\n\tif errors.As(err, &apiErr) {\n\t\treturn apiErr.ErrorCode() == \"NoSuchKey\"\n\t}\n\treturn false\n}\n\nfunc deleteOutputError(out *s3.DeleteObjectsOutput) error {\n\tif len(out.Errors) == 0 {\n\t\treturn nil\n\t}\n\n\t// Build generic error\n\tvar b strings.Builder\n\tb.WriteString(\"failed to delete files:\")\n\tfor _, err := range out.Errors {\n\t\tfmt.Fprintf(&b, \"\\n%s: %s\", aws.ToString(err.Key), aws.ToString(err.Message))\n\t}\n\treturn errors.New(b.String())\n}\n\n// parseS3DebugEnv parses the LITESTREAM_S3_DEBUG environment variable and returns\n// the corresponding AWS SDK ClientLogMode. Supports comma-separated values.\nfunc parseS3DebugEnv() aws.ClientLogMode {\n\tv := os.Getenv(\"LITESTREAM_S3_DEBUG\")\n\tif v == \"\" {\n\t\treturn 0\n\t}\n\n\tvar logMode aws.ClientLogMode\n\tfor _, mode := range strings.Split(v, \",\") {\n\t\tswitch strings.ToLower(strings.TrimSpace(mode)) {\n\t\tcase \"signing\":\n\t\t\tlogMode |= aws.LogSigning\n\t\tcase \"request\":\n\t\t\tlogMode |= aws.LogRequest\n\t\tcase \"request-with-body\":","sourceCodeStart":1841,"sourceCodeEnd":1877,"githubUrl":"https://github.com/benbjohnson/litestream/blob/4ed7a308f6271ebfd2b0a6e4b70b03011a37e4a3/s3/replica_client.go#L1841-L1877","documentation":"The S3 replica client builds this error when a multi-object delete returns per-key errors; each failed key and the AWS-reported message are appended under the 'failed to delete files:' header. It is returned by DeleteLTXFiles/DeleteAll paths when the batched delete partially or fully fails.","triggerScenarios":"Deleting LTX files on an S3 replica where the DeleteObjects API response contains entries in out.Errors (e.g. NoSuchKey, AccessDenied, SlowDown) — s3/replica_client.go:1859 formats them into one error.","commonSituations":"Objects already removed by an S3 lifecycle/expiration rule; IAM policy missing s3:DeleteObject on the bucket/prefix; S3 throttling during large batch deletes; versioned buckets with deny rules on delete markers.","solutions":["Inspect the key: message pairs listed in the error to see which objects failed and the AWS reason","Verify IAM permissions include s3:DeleteObject (and s3:DeleteObjectVersion for versioned buckets) on the litestream prefix","Adjust S3 lifecycle rules so they don't race with litestream's retention-based deletion","Retry with backoff; SlowDown/throttling errors are transient and often resolve on retry"],"exampleFix":"// before\nerr := db.Sync(ctx)\n// after\nerr := db.Sync(ctx)\nif err != nil && strings.HasPrefix(err.Error(), \"failed to delete files:\") {\n    log.Printf(\"S3 batch delete partially failed (check listed keys/IAM): %v\", err)\n}","handlingStrategy":"try-catch","validationCode":"// verify IAM delete permission before bulk deletes\niamSimulate(s3:DeleteObject, bucketArn+\"/db/*\")","typeGuard":"func isS3BatchDeleteErr(err error) bool {\n    return strings.HasPrefix(err.Error(), \"failed to delete files:\")\n}","tryCatchPattern":"if err := replicaClient.DeleteLTXFiles(ctx, files); err != nil {\n    if isS3BatchDeleteErr(err) {\n        // inspect per-key messages (NoSuchKey vs AccessDenied vs SlowDown)\n        // retry SlowDown with backoff; skip NoSuchKey; fix IAM for AccessDenied\n    }\n    return err\n}","preventionTips":["Ensure IAM policy includes s3:DeleteObject (and s3:DeleteObjectVersion for versioned buckets)","Align S3 lifecycle rules with litestream retention so keys are not already gone","Use retry with exponential backoff for SlowDown","Treat NoSuchKey entries as benign in idempotent cleanup paths"],"tags":["s3","aws","delete","batch-delete"],"backgroundTag":"batch-delete-failed","analyzedSha":"4ed7a308f6271ebfd2b0a6e4b70b03011a37e4a3","analyzedAt":"2026-09-06T18:29:25.564Z","contentChangedAt":"2026-09-06T18:29:25.564Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}