{"record":{"id":"3144e893907258a3","repo":"juicedata/juicefs","slug":"dest-read-s","errorCode":null,"errorMessage":"dest read: %s","messagePattern":"dest read: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/sync/sync.go","lineNumber":498,"sourceCode":"\t\t}()\n\t}\n\tin, err := objStor.Get(ctx, key, offset, length)\n\tif err != nil {\n\t\treturn 0, fmt.Errorf(\"dest get: %s\", err)\n\t}\n\tdefer in.Close()\n\n\tbuf := bufPool.Get().(*[]byte)\n\tdefer bufPool.Put(buf)\n\tvar chksum uint32\n\tfor left := int(length); left > 0; left -= bufferSize {\n\t\tbs := bufferSize\n\t\tif left < bufferSize {\n\t\t\tbs = left\n\t\t}\n\t\t*buf = (*buf)[:bs]\n\t\tif _, err = io.ReadFull(in, *buf); err != nil {\n\t\t\treturn 0, fmt.Errorf(\"dest read: %s\", err)\n\t\t}\n\t\tchksum = crc32.Update(chksum, crcTable, *buf)\n\t}\n\treturn chksum, nil\n}\n\nfunc calObjChksum(objStor object.ObjectStorage, key string, abort chan struct{}, obj object.Object) (uint32, error) {\n\tvar err error\n\tvar chksum uint32\n\tif obj.Size() < maxBlock {\n\t\treturn calPartChksum(objStor, key, abort, 0, obj.Size())\n\t}\n\tn := int((obj.Size()-1)/defaultPartSize) + 1\n\terrs := make(chan error, n)\n\tchksums := make([]chksumWithSz, n)\n\tfor i := 0; i < n; i++ {\n\t\tgo func(num int) {\n\t\t\tsz := int64(defaultPartSize)","sourceCodeStart":480,"sourceCodeEnd":516,"githubUrl":"https://github.com/juicedata/juicefs/blob/c9a67b23e8e08ec23ec331aa6f1675e2319e921c/pkg/sync/sync.go#L480-L516","documentation":"After obtaining the range reader, calPartChksum reads the part in bufferSize chunks with io.ReadFull to compute CRC32. A short read or read failure is wrapped as 'dest read: <err>'. Because io.ReadFull demands the full requested length, this usually means the stream ended early (fewer bytes than the expected part length) or the connection dropped mid-read.","triggerScenarios":"io.ReadFull(in, buf) returns io.ErrUnexpectedEOF/EOF (object smaller than the recorded length, e.g. changed or truncated between listing and read) or a transport error (connection reset, timeout) mid-part.","commonSituations":"Destination object truncated/resized concurrently while syncing; flaky network to object storage; misreported object size in metadata (stale listing); storage proxy dropping long transfers.","solutions":["Check the wrapped error after 'dest read:' — io.ErrUnexpectedEOF implies size mismatch, connection errors imply network issues","Re-list/re-check the destination object size; the object may have changed concurrently — re-run the sync","Improve network stability (retries, keepalives, closer endpoint/region)","If a proxy/multipart range issue is suspected, verify the storage backend honors range requests correctly"],"exampleFix":"// before (single flaky attempt, remote endpoint)\njuicefs sync s3://src https://flaky-remote/dst\n// after (closer endpoint / retry settings)\nexport AWS_REGION=us-east-1\njuicefs sync s3://src s3://dst  # same-region endpoint, stable connection","handlingStrategy":"retry","validationCode":"// verify object size matches the listing before reading the range\ninfo, err := dstStor.Head(ctx, key)\nif err == nil && info.Size < offset+length {\n\treturn fmt.Errorf(\"object %s truncated: got %d bytes, need %d\", key, info.Size, offset+length)\n}","typeGuard":null,"tryCatchPattern":"chksum, err := calPartChksum(stor, key, abort, off, length)\nif err != nil {\n\tif strings.HasPrefix(err.Error(), \"dest read:\") && isTransient(err) {\n\t\treturn retryWithBackoff(3, func() error {\n\t\t\t_, e := calPartChksum(stor, key, abort, off, length)\n\t\t\treturn e\n\t\t})\n\t}\n\treturn err\n}","preventionTips":["Re-list destination objects if they may change during sync to avoid stale sizes","Ensure stable connectivity to object storage (retries, keepalives, same-region endpoints)","Watch for io.ErrUnexpectedEOF in wrapped errors — it signals size/truncation mismatches rather than pure network issues"],"tags":["sync","io","checksum","network"],"backgroundTag":"unexpected-eof","analyzedSha":"c9a67b23e8e08ec23ec331aa6f1675e2319e921c","analyzedAt":"2026-09-06T17:55:48.476Z","contentChangedAt":"2026-09-06T17:55:48.476Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}