wavetermdev/waveterm · critical

too many flush errors (clearing entry): %w

Error message

too many flush errors (clearing entry): %w

What it means

flushToDB writes dirty cached data parts back to the SQLite filestore db via dbWriteCacheEntry. If the write fails, the error counter is incremented; once an entry has accumulated more than 3 flush errors the library gives up, clears (discards) the entire in-memory cache entry, and returns this wrapped error. The wrap means unflushed cached data was lost for that file — the library deliberately drops the entry rather than retrying indefinitely.

Source

Thrown at pkg/filestore/blockstore_cache.go:343

		FlushErrors: 0,
	}
}

func (entry *CacheEntry) flushToDB(ctx context.Context, replace bool) error {
	if entry.File == nil {
		return nil
	}
	err := dbWriteCacheEntry(ctx, entry.File, entry.DataEntries, replace)
	if ctx.Err() != nil {
		// transient error
		return ctx.Err()
	}
	if err != nil {
		flushErrorCount.Add(1)
		entry.FlushErrors++
		if entry.FlushErrors > 3 {
			entry.clear()
			return fmt.Errorf("too many flush errors (clearing entry): %w", err)
		}
		return err
	}
	// clear cache entry (data is now in db)
	entry.clear()
	return nil
}

View on GitHub (pinned to a4447c1563)

Solutions

1. Check the wrapped cause: if 'database is locked' or 'disk I/O error', free disk space / resolve db contention before more writes.
2. Assume the affected cached file data was discarded — re-read or re-write the file to repopulate cache and db.
3. Check flushErrorCount metric and logs to identify whether flush failures are clustered on one zone/file or global.
4. Run PRAGMA integrity_check on the filestore sqlite db; restore from backup or delete to recreate if corrupted.
5. If writes are consistently timing out, increase _busy_timeout in MakeDB (blockstore_dbsetup.go:67) or reduce concurrent flush pressure.

Example fix

// before
if err := fs.FlushCacheEntry(ctx, zoneId, name); err != nil {
    return err
}
// after
if err := fs.FlushCacheEntry(ctx, zoneId, name); err != nil {
    if strings.Contains(err.Error(), "too many flush errors") {
        log.Printf("cache entry for %s/%s discarded after repeated flush failures: %v", zoneId, name, err)
        // data was dropped from cache; rewrite to persist
        return fs.WriteFile(ctx, zoneId, name, data)
    }
    return err
}

var free uint64
if st, err := syscall.Statfs(wavebase.GetWaveDataDir(), &st); ... // check disk space before heavy writes
if st.Bavail*uint64(st.Bsize) < 100*1024*1024 {
    return fmt.Errorf("low disk space in data dir")
}

err := entry.flushToDB(ctx, false)
if err != nil && strings.Contains(err.Error(), "too many flush errors") {
    log.Printf("cached data for %s/%s was DISCARDED: %v", entry.ZoneId, entry.Name, errors.Unwrap(err))
    // treat the file as dirty: re-read/rewrite to restore
} else if err != nil {
    // transient: entry still cached, safe to retry later
}

When it happens

Trigger: dbWriteCacheEntry fails repeatedly (4+ times) for the same CacheEntry — persistent SQLite write failure such as disk full, db locked, I/O error, or corrupted db — while flushing dirty data parts. Note: if ctx itself is cancelled, the raw ctx.Err() is returned instead (treated as transient).

Common situations: Disk full on the machine hosting WAVE_DATA_DIR; SQLite WAL growth under sustained lock contention; a corrupted filestore db; long-running offline connection cache filling up while the db is unavailable.

Related errors

• error creating block term file: %w
• flush already in progress
• error flushing cache entry[%v]: %v
• error getting file: %w
• opening db: %w

AI-assisted analysis of wavetermdev/waveterm@a4447c1563 (2026-09-01). Data as JSON: /api/errors/5d7df8b8aceffc54. Report an issue: GitHub.