What causes "dest get: %s" in juicedata/juicefs?

Read the wrapped error after 'dest get:' for the concrete backend cause (403/404/timeout etc.)

dest get: %s

Error message

dest get: %s

What it means

calPartChksum fetches a byte range of an object from the (destination-side) object storage to compute its CRC32. If objStor.Get fails it wraps the backend error as 'dest get: <err>'. The root cause is in the wrapped object-storage error (auth, missing key, network, permissions).

Source

Thrown at pkg/sync/sync.go:484

	}
	logger.Debugf("Copied permissions (%s:%s:%s) for %s in %s", fi.Owner(), fi.Group(), fi.Mode(), key, time.Since(start))
}

func calPartChksum(objStor object.ObjectStorage, key string, abort chan struct{}, offset, length int64) (uint32, error) {
	if limiter != nil {
		limiter.Wait(length)
	}
	select {
	case <-abort:
		return 0, fmt.Errorf("aborted")
	case concurrent <- 1:
		defer func() {
			<-concurrent
		}()
	}
	in, err := objStor.Get(ctx, key, offset, length)
	if err != nil {
		return 0, fmt.Errorf("dest get: %s", err)
	}
	defer in.Close()

	buf := bufPool.Get().(*[]byte)
	defer bufPool.Put(buf)
	var chksum uint32
	for left := int(length); left > 0; left -= bufferSize {
		bs := bufferSize
		if left < bufferSize {
			bs = left
		}
		*buf = (*buf)[:bs]
		if _, err = io.ReadFull(in, *buf); err != nil {
			return 0, fmt.Errorf("dest read: %s", err)
		}
		chksum = crc32.Update(chksum, crcTable, *buf)
	}
	return chksum, nil

View on GitHub (pinned to c9a67b23e8)

Solutions

Read the wrapped error after 'dest get:' for the concrete backend cause (403/404/timeout etc.)
Verify destination storage credentials, region, and endpoint in the sync arguments
Check the object still exists in the destination (it may have been deleted concurrently by another process or lifecycle rule)
Test connectivity: curl/GET the same object URL from the sync host; check firewall/proxy settings

Example fix

// before
juicefs sync s3://src s3://dst --storage-class WRONG
// -> dest get: InvalidStorageClass ...
// after
juicefs sync s3://src s3://dst  # valid/omitted storage options
// or fix AWS credentials:
export AWS_ACCESS_KEY_ID=... AWS_SECRET_ACCESS_KEY=...

Defensive patterns

Strategy: retry

Validate before calling

// probe the destination object before checksumming
if _, err := dstStor.Head(ctx, key); err != nil {
	return fmt.Errorf("destination object %s unreachable before sync: %w", key, err)
}

Try / catch

chksum, err := calPartChksum(stor, key, abort, off, length)
if err != nil {
	if strings.HasPrefix(err.Error(), "dest get:") {
		return retryWithBackoff(3, func() error {
			_, err := calPartChksum(stor, key, abort, off, length)
			return err
		})
	}
	return err
}

Prevention

Validate destination credentials/endpoint with a HEAD/GET before large syncs
Guard against concurrent deletions/lifecycle rules on the destination bucket
Use same-region endpoints to reduce transient network failures

When it happens

Trigger: objStor.Get(ctx, key, offset, length) returns an error: the object/key doesn't exist, credentials are wrong/expired, network timeout to the storage endpoint, or the requested range is invalid.

Common situations: Destination object deleted between listing and checksum; wrong credentials or region configured for the destination; endpoint unreachable (VPN/firewall); presigned-token expiry during long syncs.

Related errors

AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06). Data as JSON: /api/errors/ae02893a297ae1ef. Report an issue: GitHub.