crowdsecurity/crowdsec · warning

pull already in progress

Error message

pull already in progress

What it means

PullTop serializes CAPI pulls using a buffered channel (isPulling) of capacity 1 as a mutex. If a pull is already running, acquiring the slot falls into the default branch and returns this error instead of queueing a second concurrent pull.

Source

Thrown at pkg/apiserver/apic.go:578

}

// we receive a list of decisions and links for blocklist and we need to create a list of alerts :
// one alert for "community blocklist"
// one alert per list we're subscribed to
func (a *apic) PullTop(ctx context.Context, forcePull bool) error {
	var err error

	hasPulledAllowlists := false

	// A mutex with TryLock would be a bit simpler
	// But go does not guarantee that TryLock will be able to acquire the lock even if it is available
	select {
	case a.isPulling <- true:
		defer func() {
			<-a.isPulling
		}()
	default:
		return errors.New("pull already in progress")
	}

	if !forcePull {
		if lastPullIsOld, err := a.CAPIPullIsOld(ctx); err != nil {
			return err
		} else if !lastPullIsOld {
			return nil
		}
	}

	log.Debug("Acquiring lock for pullCAPI")

	err = a.dbClient.AcquirePullCAPILock(ctx)
	if a.dbClient.IsLocked(err) {
		log.Info("PullCAPI is already running, skipping")
		return nil
	}

View on GitHub (pinned to 909b515798)

Solutions

1. Wait and retry the pull after the current one finishes (backoff/retry loop on this error).
2. Increase the pull interval in config so pulls don't overlap.
3. Check for stuck pulls: if the first pull never completes (hung network), restart crowdsec; consider increasing HTTP timeouts.

Example fix

// before
if err := apic.PullTop(ctx, true); err != nil { return err }
// after
if err := apic.PullTop(ctx, true); err != nil {
    if err.Error() == "pull already in progress" {
        log.Info("CAPI pull already running, skipping")
        return nil
    }
    return err
}

// no pre-call validation possible; check an external in-progress flag if you manage one
if pulling.Swap(true) { skip }

err := apic.PullTop(ctx, false)
if errors.Is(err, errPullInProgress) || strings.Contains(err.Error(), "already in progress") {
    time.Sleep(time.Minute)
    return apic.PullTop(ctx, false)
}

When it happens

Trigger: Calling PullTop (directly or via Pull / ManagementCmd) while a previous CAPI pull is still in flight; e.g. a manual cscli update triggering a pull while the periodic pull loop is running, or two overlapping pulls with forcePull.

Common situations: Slow CAPI downloads (large community blocklist, slow network) overlapping the next scheduled pull; crowdscaled deployments where a signal-triggered pull races the timer; tests reusing an APIC instance without cleanup.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• while checking if we should force pull blocklist %s: %w
• while getting decisions from blocklist %s: %w
• blocklist URL is nil
• token not found in DB
• unable to parse token

AI-assisted analysis of crowdsecurity/crowdsec@909b515798 (2026-09-06). Data as JSON: /api/errors/d4bd73c07ee0e55f. Report an issue: GitHub.