{"record":{"id":"05d2aa7bd47e8772","repo":"juicedata/juicefs","slug":"object-size-d-is-too-large-to-copy","errorCode":null,"errorMessage":"object size %d is too large to copy","messagePattern":"object size (.+?) is too large to copy","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/sync/sync.go","lineNumber":927,"sourceCode":"\n\terr = try(3, func() error { return dst.CompleteUpload(ctx, tmpkey, up.UploadID, parts) })\n\tif err != nil {\n\t\tdst.AbortUpload(ctx, tmpkey, up.UploadID)\n\t\treturn nil, 0, fmt.Errorf(\"multipart: %s\", err)\n\t}\n\tvar part *object.Part\n\terr = try(3, func() error {\n\t\tpart, err = dst.UploadPartCopy(ctx, key, upload.UploadID, num+1, tmpkey, 0, size)\n\t\treturn err\n\t})\n\t_ = dst.Delete(ctx, tmpkey)\n\treturn part, tmpChksum, err\n}\n\nfunc doCopyMultiple(src, dst object.ObjectStorage, key string, size int64, mtime time.Time, upload *object.MultipartUpload, calChksum bool, uploads multipartUploads) (uint32, error) {\n\tlimits := dst.Limits()\n\tif size > limits.MaxPartSize*int64(upload.MaxCount) {\n\t\treturn 0, fmt.Errorf(\"object size %d is too large to copy\", size)\n\t}\n\tpartSize := choosePartSize(upload, size)\n\tn := int((size-1)/partSize) + 1\n\tlogger.Debugf(\"Copying data of %s as %d parts (size: %d): %s\", key, n, partSize, upload.UploadID)\n\tabort := make(chan struct{})\n\tparts := make([]*object.Part, n)\n\terrs := make(chan error, n)\n\tchksums := make([]chksumWithSz, n)\n\tvar err error\n\n\tvar state *multipartUploadState\n\tif uploads != nil {\n\t\tstate = uploads.EnsureMultipartUploadState(key, size, mtime, partSize, upload)\n\t}\n\n\tfor i := 0; i < n; i++ {\n\t\tsz := partSize\n\t\tif i == n-1 {","sourceCodeStart":909,"sourceCodeEnd":945,"githubUrl":"https://github.com/juicedata/juicefs/blob/c9a67b23e8e08ec23ec331aa6f1675e2319e921c/pkg/sync/sync.go#L909-L945","documentation":"Raised by doCopyMultiple before any transfer starts: the object size exceeds what a single multipart copy can hold on the destination (limits.MaxPartSize * upload.MaxCount). The copy is rejected up front rather than failing partway.","triggerScenarios":"Calling sync/copy of an object whose size > dst.Limits().MaxPartSize * upload.MaxCount; e.g. a 6 TB object to a store with 5 GB max part size and 10000 max parts.","commonSituations":"Copying very large objects (videos, backups, DB dumps) between object stores with stricter limits than the source (e.g. to a store with 1 GB part limit); misconfigured MaxPartSize/MaxCount in a custom object storage implementation.","solutions":["Reduce the object size or split it into smaller objects before syncing.","Sync to a destination that supports larger multipart limits (higher MaxPartSize or MaxCount).","If you implemented a custom object.ObjectStorage, fix its Limits() to return accurate MaxPartSize/MaxCount values.","Update the destination backend SDK limits if they changed (e.g. newer S3 supports larger part counts)."],"exampleFix":"// custom backend returning wrong limits\nfunc (s *store) Limits() object.Limits { return object.Limits{MaxPartSize: 1 << 20, MaxCount: 1000} }\n// after\nfunc (s *store) Limits() object.Limits { return object.Limits{MaxPartSize: 5 << 30, MaxCount: 10000} }","handlingStrategy":"validation","validationCode":"func fitsMultipart(dst object.ObjectStorage, upload *object.MultipartUpload, size int64) error {\n  l := dst.Limits()\n  if size > l.MaxPartSize*int64(upload.MaxCount) {\n    return fmt.Errorf(\"object size %d exceeds dst multipart capacity %d\", size, l.MaxPartSize*int64(upload.MaxCount))\n  }\n  return nil\n}","typeGuard":"func tooLargeForDst(dst object.ObjectStorage, upload *object.MultipartUpload, size int64) bool {\n  l := dst.Limits()\n  return size > l.MaxPartSize*int64(upload.MaxCount)\n}","tryCatchPattern":"if err := fitsMultipart(dst, upload, size); err != nil {\n  return fmt.Errorf(\"split object before sync: %w\", err)\n}","preventionTips":["Check dst.Limits() against the largest objects you plan to sync.","Split huge files (archives, dumps) before syncing to limited backends.","If implementing object.ObjectStorage, return accurate Limits() values."],"tags":["sync","object-storage","size-limit"],"backgroundTag":"payload-too-large","analyzedSha":"c9a67b23e8e08ec23ec331aa6f1675e2319e921c","analyzedAt":"2026-09-06T17:55:48.476Z","contentChangedAt":"2026-09-06T17:55:48.476Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}