{"record":{"id":"38ba7fedda4de551","repo":"Netflix/chaosmonkey","slug":"database-migration-failed","errorCode":null,"errorMessage":"database migration failed","messagePattern":"database migration failed","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"mysql/mysql.go","lineNumber":452,"sourceCode":"\t_, err = tx.Exec(\"INSERT INTO terminations (app, account, stack, cluster, region, asg, instance_id, killed_at, leashed) VALUES (?, ?, ?, ?, ?, ?, ?, ?, ?)\",\n\t\ti.AppName(), i.AccountName(), i.StackName(), i.ClusterName(), i.RegionName(), i.ASGName(), i.ID(), term.Time.In(time.UTC), term.Leashed)\n\n\treturn err\n}\n\nvar migrationSource = &migrate.AssetMigrationSource{\n\tAsset:    migration.Asset,\n\tAssetDir: migration.AssetDir,\n\tDir:      \"migration/mysql\",\n}\n\nvar databaseDialect = \"mysql\"\n\n// Migrate upgrades a database to the latest database schema version.\nfunc Migrate(mysqlDb MySQL) error {\n\tmigrationCount, err := migrate.Exec(mysqlDb.db, databaseDialect, migrationSource, migrate.Up)\n\tif err != nil {\n\t\treturn errors.Wrap(err, \"database migration failed\")\n\t}\n\tlog.Println(\"Successfully applied database migrations. Number of migrations applied: \", migrationCount)\n\n\treturn nil\n}\n","sourceCodeStart":434,"sourceCodeEnd":458,"githubUrl":"https://github.com/Netflix/chaosmonkey/blob/eaa28fb761c0ebe8644d1333e5d164e9cc3071e9/mysql/mysql.go#L434-L458","documentation":"Migrate applies the embedded SQL migrations to the MySQL database using migrate.Exec. If any migration fails to execute, the error is wrapped as 'database migration failed'. The database is therefore left at whatever schema version it had, and the monkey will not start correctly.","triggerScenarios":"migrate.Exec returns an error: SQL syntax/schema incompatibility, the DB user lacks privileges to create/alter tables, connectivity drops mid-migration, or a prior failed migration left the schema version tracking inconsistent.","commonSituations":"First run against a fresh database with a restricted DB user; MySQL version incompatible with migration SQL; partially applied migration from an interrupted earlier run; wrong database pointed at by config.","solutions":["Read the wrapped cause (%+v) for the exact failing migration and SQL error","Verify the DB user has CREATE/ALTER/INSERT privileges on the target schema","Check the schema_migrations table for a partially applied migration and repair/re-run manually","Confirm the configured host/database is the intended one and reachable","Test migrations against a staging copy of the database first"],"exampleFix":"// before\nGRANT SELECT, INSERT, UPDATE, DELETE ON chaosmonkey.* TO 'monkey'@'%';\n// after\nGRANT SELECT, INSERT, UPDATE, DELETE, CREATE, ALTER, INDEX, DROP ON chaosmonkey.* TO 'monkey'@'%';","handlingStrategy":"try-catch","validationCode":"// pre-check privileges and connectivity before Migrate\nif err := mysqlDb.Ping(); err != nil {\n    return fmt.Errorf(\"db unreachable before migration: %w\", err)\n}\n// verify the user can DDL\ntxn, _ := mysqlDb.Begin()\nif _, err := txn.Exec(\"CREATE TABLE IF NOT EXISTS _mig_probe (id int)\"); err != nil {\n    txn.Rollback()\n    return fmt.Errorf(\"db user lacks DDL privileges: %w\", err)\n}\ntxn.Rollback()","typeGuard":null,"tryCatchPattern":"if err := mysql.Migrate(store); err != nil {\n    log.Printf(\"migration failed: %+v\", err) // %+v reveals the failing migration/SQL\n    return fmt.Errorf(\"startup aborted: %w\", err)\n}","preventionTips":["Run migrations in CI against a staging database before production deploys","Grant the monkey DB user full DDL rights on its schema","Inspect schema_migrations after any interrupted deployment","Back up the database before major version upgrades","Pin MySQL server versions known to work with the migration SQL"],"tags":["go","mysql","database-migration","schema"],"backgroundTag":"database-migration-failed","analyzedSha":"eaa28fb761c0ebe8644d1333e5d164e9cc3071e9","analyzedAt":"2026-09-03T17:04:39.020Z","contentChangedAt":"2026-09-03T17:04:39.020Z","schemaVersion":2},"datasetVersion":"2026-09-08T15:18:49.778Z"}