What causes "exceeded %d busy retries" in crowdsecurity/crowdsec?

Ensure only one crowdsec process uses the database (check for duplicate agents/LAPI sharing one data dir)

exceeded %d busy retries

Error message

exceeded %d busy retries

What it means

retryOnBusy retried the wrapped database operation maxLockRetries times because SQLite kept returning SQLITE_BUSY (database is locked), and every retry still hit the busy condition. This indicates sustained lock contention on the local SQLite database — another process (cscli, a second crowdsec, LAPI) holds a write lock longer than the retry window (1s between attempts).

Source

Thrown at pkg/database/alerts.go:582

func retryOnBusy(fn func() error) error {
	for retry := range maxLockRetries {
		err := fn()
		if err == nil {
			return nil
		}

		if IsSqliteBusyError(err) {
			log.Warningf("while updating decisions, sqlite3.ErrBusy: %s, retry %d of %d", err, retry, maxLockRetries)
			time.Sleep(1 * time.Second)

			continue
		}

		return err
	}

	return fmt.Errorf("exceeded %d busy retries", maxLockRetries)
}

func (c *Client) saveAlerts(ctx context.Context, client *ent.Client, batch []alertCreatePlan) ([]string, error) {
	if len(batch) == 0 {
		log.Warningf("no alerts to create, discarded?")
		return nil, nil
	}

	// extract builders in the same order
	builders := make([]*ent.AlertCreate, len(batch))
	for i := range batch {
		if batch[i].builder == nil {
			return nil, fmt.Errorf("nil alert builder at index %d", i)
		}

		builders[i] = batch[i].builder
	}

View on GitHub (pinned to 909b515798)

Solutions

Ensure only one crowdsec process uses the database (check for duplicate agents/LAPI sharing one data dir)
Avoid running heavy cscli writes concurrently with alert ingestion; retry the failed operation once contention subsides
Find the lock holder: `fuser`/`lsof` on the sqlite db file, or check for stuck cscli/backup jobs
If contention is chronic, switch the database backend to Postgres/MySQL, which handles concurrency far better than SQLite
Check I/O health (disk saturation, NFS mounts) — slow storage makes SQLite transactions hold locks longer

Example fix

// before (diagnosis)
crowdsec -c /etc/crowdsec/local/... -- db_config.sqlite
// after (mitigation: dedicated connection tuning)
# in crowdsec config: use postgres for concurrent writers
db_config:
  type: postgresql
  host: localhost
  ...

Defensive patterns

Strategy: retry

Validate before calling

// caller-side: check DB writability and single-instance before bulk writes
if err := db.PingContext(ctx); err != nil { return err }

Try / catch

if err := retryOnBusy(op); err != nil {
    if strings.Contains(err.Error(), "exceeded") {
        // all busy retries exhausted: back off longer or move to a client-server DB
    }
    return err
}

Prevention

Never run two crowdsec instances against one SQLite file
Serialize cscli write commands with ingestion (or accept occasional retries)
Consider postgresql/mysql for multi-writer deployments
Keep the DB on fast local storage; avoid NFS

When it happens

Trigger: Concurrent writes to the same SQLite DB while alerts/decisions are being saved: running cscli commands while crowdsec is writing, multiple crowdsec instances sharing one DB, long transactions from other clients, or a slow/hung writer holding the lock.

Common situations: Running `cscli decisions list/add/delete` at the same time the alert flush is saving a large batch; two crowdsec processes pointed at the same data dir by mistake; backups or other tools holding a read lock on crowdsec.db; disk I/O stall making transactions slow.

Related errors

AI-assisted analysis of crowdsecurity/crowdsec@909b515798 (2026-09-06). Data as JSON: /api/errors/2d2bc6eeb03c6d71. Report an issue: GitHub.