juicedata/juicefs · error
dest read: %s
Error message
dest read: %s
What it means
After obtaining the range reader, calPartChksum reads the part in bufferSize chunks with io.ReadFull to compute CRC32. A short read or read failure is wrapped as 'dest read: <err>'. Because io.ReadFull demands the full requested length, this usually means the stream ended early (fewer bytes than the expected part length) or the connection dropped mid-read.
Source
Thrown at pkg/sync/sync.go:498
}()
}
in, err := objStor.Get(ctx, key, offset, length)
if err != nil {
return 0, fmt.Errorf("dest get: %s", err)
}
defer in.Close()
buf := bufPool.Get().(*[]byte)
defer bufPool.Put(buf)
var chksum uint32
for left := int(length); left > 0; left -= bufferSize {
bs := bufferSize
if left < bufferSize {
bs = left
}
*buf = (*buf)[:bs]
if _, err = io.ReadFull(in, *buf); err != nil {
return 0, fmt.Errorf("dest read: %s", err)
}
chksum = crc32.Update(chksum, crcTable, *buf)
}
return chksum, nil
}
func calObjChksum(objStor object.ObjectStorage, key string, abort chan struct{}, obj object.Object) (uint32, error) {
var err error
var chksum uint32
if obj.Size() < maxBlock {
return calPartChksum(objStor, key, abort, 0, obj.Size())
}
n := int((obj.Size()-1)/defaultPartSize) + 1
errs := make(chan error, n)
chksums := make([]chksumWithSz, n)
for i := 0; i < n; i++ {
go func(num int) {
sz := int64(defaultPartSize)View on GitHub (pinned to c9a67b23e8)
Solutions
- Check the wrapped error after 'dest read:' — io.ErrUnexpectedEOF implies size mismatch, connection errors imply network issues
- Re-list/re-check the destination object size; the object may have changed concurrently — re-run the sync
- Improve network stability (retries, keepalives, closer endpoint/region)
- If a proxy/multipart range issue is suspected, verify the storage backend honors range requests correctly
Example fix
// before (single flaky attempt, remote endpoint) juicefs sync s3://src https://flaky-remote/dst // after (closer endpoint / retry settings) export AWS_REGION=us-east-1 juicefs sync s3://src s3://dst # same-region endpoint, stable connection
Defensive patterns
Strategy: retry
Validate before calling
// verify object size matches the listing before reading the range
info, err := dstStor.Head(ctx, key)
if err == nil && info.Size < offset+length {
return fmt.Errorf("object %s truncated: got %d bytes, need %d", key, info.Size, offset+length)
} Try / catch
chksum, err := calPartChksum(stor, key, abort, off, length)
if err != nil {
if strings.HasPrefix(err.Error(), "dest read:") && isTransient(err) {
return retryWithBackoff(3, func() error {
_, e := calPartChksum(stor, key, abort, off, length)
return e
})
}
return err
} Prevention
- Re-list destination objects if they may change during sync to avoid stale sizes
- Ensure stable connectivity to object storage (retries, keepalives, same-region endpoints)
- Watch for io.ErrUnexpectedEOF in wrapped errors — it signals size/truncation mismatches rather than pure network issues
When it happens
Trigger: io.ReadFull(in, buf) returns io.ErrUnexpectedEOF/EOF (object smaller than the recorded length, e.g. changed or truncated between listing and read) or a transport error (connection reset, timeout) mid-part.
Common situations: Destination object truncated/resized concurrently while syncing; flaky network to object storage; misreported object size in metadata (stale listing); storage proxy dropping long transfers.
Related errors
AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06).
Data as JSON: /api/errors/3144e893907258a3.
Report an issue: GitHub.