juicedata/juicefs · error

object size %d is too large to copy

Error message

object size %d is too large to copy

What it means

Raised by doCopyMultiple before any transfer starts: the object size exceeds what a single multipart copy can hold on the destination (limits.MaxPartSize * upload.MaxCount). The copy is rejected up front rather than failing partway.

Source

Thrown at pkg/sync/sync.go:927

	err = try(3, func() error { return dst.CompleteUpload(ctx, tmpkey, up.UploadID, parts) })
	if err != nil {
		dst.AbortUpload(ctx, tmpkey, up.UploadID)
		return nil, 0, fmt.Errorf("multipart: %s", err)
	}
	var part *object.Part
	err = try(3, func() error {
		part, err = dst.UploadPartCopy(ctx, key, upload.UploadID, num+1, tmpkey, 0, size)
		return err
	})
	_ = dst.Delete(ctx, tmpkey)
	return part, tmpChksum, err
}

func doCopyMultiple(src, dst object.ObjectStorage, key string, size int64, mtime time.Time, upload *object.MultipartUpload, calChksum bool, uploads multipartUploads) (uint32, error) {
	limits := dst.Limits()
	if size > limits.MaxPartSize*int64(upload.MaxCount) {
		return 0, fmt.Errorf("object size %d is too large to copy", size)
	}
	partSize := choosePartSize(upload, size)
	n := int((size-1)/partSize) + 1
	logger.Debugf("Copying data of %s as %d parts (size: %d): %s", key, n, partSize, upload.UploadID)
	abort := make(chan struct{})
	parts := make([]*object.Part, n)
	errs := make(chan error, n)
	chksums := make([]chksumWithSz, n)
	var err error

	var state *multipartUploadState
	if uploads != nil {
		state = uploads.EnsureMultipartUploadState(key, size, mtime, partSize, upload)
	}

	for i := 0; i < n; i++ {
		sz := partSize
		if i == n-1 {

View on GitHub (pinned to c9a67b23e8)

Solutions

1. Reduce the object size or split it into smaller objects before syncing.
2. Sync to a destination that supports larger multipart limits (higher MaxPartSize or MaxCount).
3. If you implemented a custom object.ObjectStorage, fix its Limits() to return accurate MaxPartSize/MaxCount values.
4. Update the destination backend SDK limits if they changed (e.g. newer S3 supports larger part counts).

Example fix

// custom backend returning wrong limits
func (s *store) Limits() object.Limits { return object.Limits{MaxPartSize: 1 << 20, MaxCount: 1000} }
// after
func (s *store) Limits() object.Limits { return object.Limits{MaxPartSize: 5 << 30, MaxCount: 10000} }

func fitsMultipart(dst object.ObjectStorage, upload *object.MultipartUpload, size int64) error {
  l := dst.Limits()
  if size > l.MaxPartSize*int64(upload.MaxCount) {
    return fmt.Errorf("object size %d exceeds dst multipart capacity %d", size, l.MaxPartSize*int64(upload.MaxCount))
  }
  return nil
}

func tooLargeForDst(dst object.ObjectStorage, upload *object.MultipartUpload, size int64) bool {
  l := dst.Limits()
  return size > l.MaxPartSize*int64(upload.MaxCount)
}

if err := fitsMultipart(dst, upload, size); err != nil {
  return fmt.Errorf("split object before sync: %w", err)
}

When it happens

Trigger: Calling sync/copy of an object whose size > dst.Limits().MaxPartSize * upload.MaxCount; e.g. a 6 TB object to a store with 5 GB max part size and 10000 max parts.

Common situations: Copying very large objects (videos, backups, DB dumps) between object stores with stricter limits than the source (e.g. to a store with 1 GB part limit); misconfigured MaxPartSize/MaxCount in a custom object storage implementation.

Understand the failure class

Background: payload too large / request exceeds maximum size: why libraries cap bytes and how to fix oversize payloads — this error's family across 50 libraries.

Related errors

• create %s %s: %s
• dest get: %s
• ceph: can't put empty file
• GOOGLE_CLOUD_PROJECT environment variable must be set
• object key cannot be empty

AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06). Data as JSON: /api/errors/05d2aa7bd47e8772. Report an issue: GitHub.