{"record":{"id":"afacab04897933ee","repo":"crowdsecurity/crowdsec","slug":"bulk-creating-alert-w-w","errorCode":null,"errorMessage":"bulk creating alert: %w: %w","messagePattern":"bulk creating alert: %w: %w","errorType":"exception","errorClass":"BulkError","httpStatus":null,"severity":"error","filePath":"pkg/database/alerts.go","lineNumber":603,"sourceCode":"func (c *Client) saveAlerts(ctx context.Context, client *ent.Client, batch []alertCreatePlan) ([]string, error) {\n\tif len(batch) == 0 {\n\t\tlog.Warningf(\"no alerts to create, discarded?\")\n\t\treturn nil, nil\n\t}\n\n\t// extract builders in the same order\n\tbuilders := make([]*ent.AlertCreate, len(batch))\n\tfor i := range batch {\n\t\tif batch[i].builder == nil {\n\t\t\treturn nil, fmt.Errorf(\"nil alert builder at index %d\", i)\n\t\t}\n\n\t\tbuilders[i] = batch[i].builder\n\t}\n\n\talertsCreateBulk, err := client.Alert.CreateBulk(builders...).Save(ctx)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"bulk creating alert: %w: %w\", err, BulkError)\n\t}\n\n\tret := make([]string, len(alertsCreateBulk))\n\tfor i, a := range alertsCreateBulk {\n\t\tret[i] = strconv.Itoa(a.ID)\n\n\t\td := batch[i].decisions\n\t\tif len(d) == 0 {\n\t\t\tcontinue\n\t\t}\n\n\t\tif err := slicetools.Batch(ctx, d, c.decisionBulkSize, func(ctx context.Context, d2 []*ent.Decision) error {\n\t\t\treturn retryOnBusy(func() error {\n\t\t\t\t_, err := client.Alert.Update().Where(alert.IDEQ(a.ID)).AddDecisions(d2...).Save(ctx)\n\t\t\t\treturn err\n\t\t\t})\n\t\t}); err != nil {\n\t\t\treturn nil, fmt.Errorf(\"attach decisions to alert %d: %w\", a.ID, err)","sourceCodeStart":585,"sourceCodeEnd":621,"githubUrl":"https://github.com/crowdsecurity/crowdsec/blob/909b5157986a2b2c2163300fdaef5ed01289f7d2/pkg/database/alerts.go#L585-L621","documentation":"saveAlerts performs a single ent CreateBulk insert of all alert builders inside a transaction and wraps any database error with the BulkError sentinel. This is the standard bulk-insert failure path: the database rejected one or more of the alert rows (constraint violation, size limit, connection drop, etc.), so none of the batch is saved and the caller rolls back the transaction.","triggerScenarios":"client.Alert.CreateBulk(builders...).Save(ctx) returns an error — e.g. a NOT NULL/UNIQUE constraint failure (duplicate alert UUID), a field value exceeding column size, SQLite 'database is locked'/'disk I/O error', MySQL/Postgres connection loss, or context cancellation mid-insert.","commonSituations":"Duplicate alert UUID when the same alert is submitted twice to a SQLite LAPI database; oversized scenario/message values against a fixed-size column on MySQL; sqlite database file locked by a concurrent cscli command or bouncer writes; disk full on the database volume.","solutions":["Read the wrapped underlying error (errors.Unwrap / %v of the chain) to identify the specific DB failure — constraints, lock, connection, or context","If it is a duplicate/UUID constraint, stop re-submitting the same alert or deduplicate by UUID before calling CreateAlert","If it is sqlite busy/locked, check for concurrent writers (other crowdsec/cscli processes) and ensure the DB is on a local filesystem, not NFS","If it is a connection failure, verify DB reachability (host/port/credentials in crowdsec.yaml) and retry; the transaction was rolled back so the operation is safe to repeat","If a field is too large, truncate or limit the alert payload (scenario, message) before ingestion"],"exampleFix":"// before\nids, err := client.CreateAlert(ctx, machineID, alerts)\nif err != nil {\n\treturn err\n}\n\n// after\nids, err := client.CreateAlert(ctx, machineID, dedupeByUUID(alerts))\nif err != nil {\n\tif database.IsBulkError(err) {\n\t\tlog.Errorf(\"bulk alert insert failed: %v\", errors.Unwrap(err))\n\t}\n\treturn err\n}","handlingStrategy":"retry","validationCode":"// dedupe and sanity-check alerts before submission\nseen := map[string]bool{}\nfor _, a := range alerts {\n\tif a.UUID == nil || seen[*a.UUID] {\n\t\treturn fmt.Errorf(\"missing or duplicate alert UUID\")\n\t}\n\tseen[*a.UUID] = true\n}","typeGuard":"func validAlert(a *models.Alert) bool {\n\treturn a != nil && a.UUID != nil && a.Scenario != nil && a.Source != nil && a.Source.Scope != nil && a.Source.Value != nil\n}","tryCatchPattern":"ids, err := client.CreateAlert(ctx, machineID, alerts)\nvar bulkErr *database.BulkError\nif err != nil {\n\tif errors.As(err, &bulkErr) || database.IsBulkError(err) {\n\t\tlog.Errorf(\"bulk insert rejected: %v\", err) // inspect wrapped DB cause before retry\n\t}\n}","preventionTips":["Deduplicate alerts by UUID before ingestion","Keep the crowdsec SQLite DB on a local disk, never NFS","Monitor disk space on the database volume","Validate alert payloads (all required pointer fields set) before calling CreateAlert","On MySQL/Postgres, monitor connection health and max_connections"],"tags":["database","sqlite","bulk-insert","crowdsec"],"backgroundTag":"database-write-failed","analyzedSha":"909b5157986a2b2c2163300fdaef5ed01289f7d2","analyzedAt":"2026-09-06T12:27:26.012Z","contentChangedAt":"2026-09-06T12:27:26.012Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}