{"record":{"id":"e814b829219a4ee8","repo":"juicedata/juicefs","slug":"multipart-s","errorCode":null,"errorMessage":"multipart: %s","messagePattern":"multipart: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/sync/sync.go","lineNumber":913,"sourceCode":"\t\tparts[i], chksum, err = doUploadPart(src, dst, key, off+int64(i)*partSize, sz, tmpkey, up.UploadID, i, calChksum)\n\t\tif err != nil {\n\t\t\tdst.AbortUpload(ctx, tmpkey, up.UploadID)\n\t\t\treturn nil, 0, fmt.Errorf(\"range(%d,%d): %s\", off, size, err)\n\t\t}\n\t\tif calChksum {\n\t\t\tif first {\n\t\t\t\ttmpChksum = chksum\n\t\t\t\tfirst = false\n\t\t\t} else {\n\t\t\t\ttmpChksum = crc32combine.CRC32Combine(crc32.Castagnoli, tmpChksum, chksum, sz)\n\t\t\t}\n\t\t}\n\t}\n\n\terr = try(3, func() error { return dst.CompleteUpload(ctx, tmpkey, up.UploadID, parts) })\n\tif err != nil {\n\t\tdst.AbortUpload(ctx, tmpkey, up.UploadID)\n\t\treturn nil, 0, fmt.Errorf(\"multipart: %s\", err)\n\t}\n\tvar part *object.Part\n\terr = try(3, func() error {\n\t\tpart, err = dst.UploadPartCopy(ctx, key, upload.UploadID, num+1, tmpkey, 0, size)\n\t\treturn err\n\t})\n\t_ = dst.Delete(ctx, tmpkey)\n\treturn part, tmpChksum, err\n}\n\nfunc doCopyMultiple(src, dst object.ObjectStorage, key string, size int64, mtime time.Time, upload *object.MultipartUpload, calChksum bool, uploads multipartUploads) (uint32, error) {\n\tlimits := dst.Limits()\n\tif size > limits.MaxPartSize*int64(upload.MaxCount) {\n\t\treturn 0, fmt.Errorf(\"object size %d is too large to copy\", size)\n\t}\n\tpartSize := choosePartSize(upload, size)\n\tn := int((size-1)/partSize) + 1\n\tlogger.Debugf(\"Copying data of %s as %d parts (size: %d): %s\", key, n, partSize, upload.UploadID)","sourceCodeStart":895,"sourceCodeEnd":931,"githubUrl":"https://github.com/juicedata/juicefs/blob/c9a67b23e8e08ec23ec331aa6f1675e2319e921c/pkg/sync/sync.go#L895-L931","documentation":"Wraps a failure of CompleteUpload after all parts of a multipart copy were uploaded. When dst.CompleteUpload fails 3 times (via try), the multipart upload is aborted and the error is reported as 'multipart: <err>'. The upload produced no object; the caller must retry the whole copy.","triggerScenarios":"dst.CompleteUpload(ctx, tmpkey, up.UploadID, parts) returns an error (retried 3 times) — typically because one or more uploaded parts are missing/invalid (ETag mismatch), the upload ID was expired, or the destination rejected the complete call.","commonSituations":"Long uploads where the destination expired the multipart upload ID (e.g. S3 lifecycle rule aborting incomplete multipart uploads); part list mismatch after a retried/aborted part; destination service outage during CompleteUpload.","solutions":["Check the wrapped error for expired/aborted upload ID and rerun sync — a fresh upload will be started automatically.","Remove or extend S3 'abort incomplete multipart upload' lifecycle rules on the destination bucket for large copies.","Verify part ETags/validity: avoid concurrent writers to the same destination key.","Retry during off-peak hours or with fewer --threads if the destination rejects the complete call under load."],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// ensure no lifecycle rule aborts incomplete multipart uploads too early\nrules := getS3LifecycleRules(dstBucket)\nfor _, r := range rules {\n  if r.AbortIncompleteMultipartUpload != nil && r.AbortIncompleteMultipartUpload.DaysAfterInitiation < expectedUploadHours*24 {\n    return fmt.Errorf(\"lifecycle rule would abort upload\")\n  }\n}","typeGuard":null,"tryCatchPattern":"if err := sync(...); err != nil {\n  if strings.HasPrefix(err.Error(), \"multipart:\") {\n    logger.Warnf(\"multipart complete failed, retrying sync: %v\", err)\n    return retryWithBackoff(sync, 3)\n  }\n  return err\n}","preventionTips":["Disable/extend 'abort incomplete multipart upload' lifecycle rules on the destination for long copies.","Make syncs exclusive per key; avoid concurrent writers to the same destination object.","Keep upload sessions short enough to finish before destination upload-ID expiry."],"tags":["sync","multipart-upload","s3"],"backgroundTag":"http-error-response","analyzedSha":"c9a67b23e8e08ec23ec331aa6f1675e2319e921c","analyzedAt":"2026-09-06T17:55:48.476Z","contentChangedAt":"2026-09-06T17:55:48.476Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}