juicedata/juicefs · error
failed to clean up sid %d
Error message
failed to clean up sid %d
What it means
Returned by dbMeta.doCleanStaleSession when any step of releasing a dead session's resources failed: deleting its flock/plock rows, scanning its sustained inodes, or deleting sustained inodes. When `fail` is set, the session row itself is deliberately NOT deleted so cleanup can be retried; only when everything succeeds is the session record removed.
Source
Thrown at pkg/meta/sql.go:3192
var sus []sustained
err = m.simpleTxn(Background(), func(ses *xorm.Session) error {
sus = nil
return ses.Find(&sus, &sustained{Sid: sid})
})
if err != nil {
logger.Warnf("Scan sustained with sid %d: %s", sid, err)
fail = true
} else {
for _, su := range sus {
if err = m.doDeleteSustainedInode(sid, su.Inode); err != nil {
logger.Warnf("Delete sustained inode %d of sid %d: %s", su.Inode, sid, err)
fail = true
}
}
}
if fail {
return fmt.Errorf("failed to clean up sid %d", sid)
} else {
return m.txn(func(s *xorm.Session) error {
var deleted bool
if n, err := s.Delete(&session2{Sid: sid}); err != nil {
return err
} else if n == 1 {
deleted = true
}
ok, err := s.IsTableExist(&session{})
if err != nil {
return err
}
if ok {
n, err := s.Delete(&session{Sid: sid})
if err != nil {
return err
}
deleted = deleted || n == 1View on GitHub (pinned to c9a67b23e8)
Solutions
- Check the preceding Warnf log lines ('Delete flock/plock with sid', 'Scan sustained with sid', 'Delete sustained inode ... of sid') to find which step failed and fix its root cause.
- Verify metadata DB and object storage health; cleanup is retried on the next scan once healthy.
- If a specific sustained inode keeps failing, run `juicefs fsck` on that inode and resolve data/metadata inconsistency.
- Reduce cleanup pressure (stagger client restarts) to avoid lock timeouts during mass cleanup.
Example fix
// before: cleanup logs 'Delete sustained inode 100 of sid 42: ...' then "failed to clean up sid 42"; session row kept // after: fix object storage access, let the next cleanup pass succeed juicefs gc sqlite3://test.db # re-run after restoring DB/object-store health; session2 row is then deleted
Defensive patterns
Strategy: retry
Validate before calling
// preflight: metadata DB and object storage reachable before triggering cleanup
if err := db.Ping(); err != nil { return err }
if err := objStore.Head(ctx, "__probe"); err != nil { return err } Try / catch
if err := cleanupStaleSessions(); err != nil {
if strings.Contains(err.Error(), "failed to clean up sid") {
logger.Warnf("cleanup deferred, will retry next scan: %v", err)
time.Sleep(time.Minute)
return cleanupStaleSessions() // session row was kept, retry is safe
}
return err
} Prevention
- Ensure object storage credentials/health before running `juicefs gc` or letting cleanup run.
- Stagger mass client shutdowns to avoid cleanup storms against the DB.
- Investigate per-inode Warnf logs when the same sid repeatedly fails cleanup.
When it happens
Trigger: Stale-session cleanup (heartbeat expiry, `juicefs gc`) on a dead sid while the database is degraded: connection failures deleting flock/plock, read errors scanning sustained, or doDeleteSustainedInode failing (e.g. chunk deletion errors, txn conflicts) for any of the session's open inodes.
Common situations: Metadata DB flapping during a mass client crash (network cut kills many clients at once, cleanup storms the DB); blob store (object storage) errors while deleting chunks of sustained inodes; lock-wait timeouts on busy tables during cleanup.
Related errors
AI-assisted analysis of juicedata/juicefs@c9a67b23e8 (2026-09-06).
Data as JSON: /api/errors/8a4d4f33624a803e.
Report an issue: GitHub.