juicedata/juicefs · warning
object %s was removed during sync
Error message
object %s was removed during sync
What it means
Raised by checkChange when the SOURCE object no longer exists at verification time: errors.Is(err, os.ErrNotExist) from src.Head. The object was listed at the start of the sync but deleted before the check ran, so a consistent copy can no longer be guaranteed.
Source
Thrown at pkg/sync/sync.go:1258
equal := cur.Size() == obj.Size()
if equal && !cur.Mtime().Equal(obj.Mtime()) {
// Head of an object may not return the millisecond part of mtime as List
equal = cur.Mtime().Unix() == obj.Mtime().Unix() && cur.Mtime().UnixMilli()%1000 == 0
}
if !equal {
return fmt.Errorf("%s changed during sync. Original: size=%d, mtime=%s; Current: size=%d, mtime=%s",
cur.Key(), obj.Size(), obj.Mtime(), cur.Size(), cur.Mtime())
}
if dstObj, err := dst.Head(ctx, key); err == nil {
if cur.Size() != dstObj.Size() {
return fmt.Errorf("copied %s size mismatch: original=%d, current=%d", key, obj.Size(), dstObj.Size())
}
return nil
} else {
return fmt.Errorf("check %s in %s: %s", key, dst, err)
}
} else if errors.Is(err, os.ErrNotExist) {
return fmt.Errorf("object %s was removed during sync", key)
} else {
return fmt.Errorf("check %s in %s: %s", key, src, err)
}
}
func copyLink(src object.ObjectStorage, dst object.ObjectStorage, key string) error {
var p string
var err error
if p, err = src.(object.SupportSymlink).Readlink(key); err != nil {
return err
}
return try(3, func() (err error) {
// TODO: use relative path based on option
if err := dst.(object.SupportSymlink).Symlink(p, key); err != nil {
if info, err := dst.Head(ctx, key); err == nil && info.IsSymlink() {
if cPath, err2 := dst.(object.SupportSymlink).Readlink(key); err2 == nil && p == cPath {
return nil
}View on GitHub (pinned to c9a67b23e8)
Solutions
- Rerun sync — the next listing won't include the deleted object; this error for a genuinely deleted object is usually harmless and self-healing.
- Exclude ephemeral prefixes (tmp/, *.log rotation targets) with --exclude to avoid racing deleters.
- Quiesce or reschedule cleanup jobs and source lifecycle rules during the sync window.
- If deletions should propagate to the destination, run sync with a delete-propagation option instead of treating it as an error.
Example fix
// exclude ephemeral files that get deleted mid-sync juicefs sync src://bucket dst://bucket --exclude 'tmp/*' --exclude '*.tmp'
Defensive patterns
Strategy: try-catch
Validate before calling
// detect racy sources up front: list twice and compare presence
a, _ := listKeys(src, prefix)
b, _ := listKeys(src, prefix)
if len(diff(a, b)) > 0 { log.Print("source keys are being deleted concurrently") } Try / catch
if err := sync(...); err != nil {
if strings.Contains(err.Error(), "was removed during sync") {
logger.Warnf("skipping object deleted mid-sync: %v", err)
return nil // benign for append/ephemeral sources
}
return err
} Prevention
- Exclude ephemeral/tmp prefixes with --exclude.
- Pause cleanup jobs and lifecycle expiration during the sync window.
- Treat single occurrences as benign; rerun sync to get a clean pass.
- Use delete-propagation semantics explicitly if deletions should be mirrored.
When it happens
Trigger: sync lists objects, copies one, then calls checkChange; src.Head returns os.ErrNotExist because the object was deleted concurrently during the sync run.
Common situations: Log rotation/cleanup jobs deleting files from the source while sync runs; lifecycle expiration rules on the source bucket; a second sync or user deleting keys concurrently; syncing directories with ephemeral/temp files.
Understand the failure class
Background: "File not found" and ENOENT errors: why libraries can't find a file that should exist — this error's family across 50 libraries.
Related errors
AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06).
Data as JSON: /api/errors/12887eea78fbad81.
Report an issue: GitHub.