{"record":{"id":"0552c97fc45c4713","repo":"crowdsecurity/crowdsec","slug":"unable-to-insert-bulk","errorCode":null,"errorMessage":"unable to insert bulk","messagePattern":"unable to insert bulk","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/database/errors.go","lineNumber":17,"sourceCode":"package database\n\nimport \"errors\"\n\nvar (\n\tUserExists        = errors.New(\"user already exist\")\n\tUserNotExists     = errors.New(\"user doesn't exist\")\n\tHashError         = errors.New(\"unable to hash\")\n\tInsertFail        = errors.New(\"unable to insert row\")\n\tQueryFail         = errors.New(\"unable to query\")\n\tUpdateFail        = errors.New(\"unable to update\")\n\tDeleteFail        = errors.New(\"unable to delete\")\n\tItemNotFound      = errors.New(\"object not found\")\n\tParseTimeFail     = errors.New(\"unable to parse time\")\n\tParseDurationFail = errors.New(\"unable to parse duration\")\n\tMarshalFail       = errors.New(\"unable to serialize\")\n\tBulkError         = errors.New(\"unable to insert bulk\")\n\tParseType         = errors.New(\"unable to parse type\")\n\tInvalidIPOrRange  = errors.New(\"invalid ip address / range\")\n\tInvalidFilter     = errors.New(\"invalid filter\")\n)\n","sourceCodeStart":1,"sourceCodeEnd":22,"githubUrl":"https://github.com/crowdsecurity/crowdsec/blob/909b5157986a2b2c2163300fdaef5ed01289f7d2/pkg/database/errors.go#L1-L22","documentation":"BulkError is a sentinel error returned when persisting an alert (and its events/decisions) into the database fails: the alert builder Save, the transaction creation, or a bulk insert fails. It wraps the underlying ent/SQL error so errors.Is matching still works.","triggerScenarios":"alertB.Save(ctx) failing on constraint violations; c.Ent.Tx(ctx) failing to open a transaction; createAlertBatch/saveAlerts bulk-inserting events or decisions and hitting an error; UpdateCommunityBlocklist bulk upserts failing.","commonSituations":"DB connection dropped mid-insert (MySQL 'server has gone away'), unique constraint collisions on concurrent alert ingestion, disk-full database, oversized batch exceeding max_allowed_packet on MySQL.","solutions":["Read the wrapped inner error for the true cause (constraint, connection, packet size).","Retry on transient connection errors; crowdsec's DB driver may need higher timeouts in db_config.","Reduce batch size (fewer alerts/events per call) to stay under DB packet limits.","Check disk space and DB health; verify schema is migrated."],"exampleFix":"// before\nfor _, alert := range thousandsOfAlerts {\n    client.CreateAlert(ctx, alert) // one giant bulk insert fails\n}\n// after\nfor chunk := range slices.Chunk(thousandsOfAlerts, 50) {\n    if _, err := client.CreateAlerts(ctx, chunk); err != nil {\n        return fmt.Errorf(\"batch: %w\", err)\n    }\n}","handlingStrategy":"try-catch","validationCode":"// Go: keep batches bounded and DB reachable before bulk writes\nif len(alerts) > 100 { alerts = alerts[:100] }\nif err := db.PingContext(ctx); err != nil { return err }","typeGuard":"func isBulkError(err error) bool { return errors.Is(err, database.BulkError) }","tryCatchPattern":"if err := saveAlerts(ctx, alerts); err != nil {\n    if errors.Is(err, database.BulkError) {\n        log.Warnf(\"bulk save failed: %v\", err)\n        return retryWithBackoff(alerts)\n    }\n    return err\n}","preventionTips":["Batch inserts in modest sizes (avoid max_allowed_packet limits)","Add retry with backoff for transient connection errors","Monitor DB disk space and connection health"],"tags":["database","crowdsec","insert","bulk"],"backgroundTag":"database-write-failed","analyzedSha":"909b5157986a2b2c2163300fdaef5ed01289f7d2","analyzedAt":"2026-09-06T12:27:26.012Z","contentChangedAt":"2026-09-06T12:27:26.012Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}