{"record":{"id":"947df0edeb3c8553","repo":"vitessio/vitess","slug":"failed-query-commit-err-s","errorCode":null,"errorMessage":"failed query COMMIT, err: %s","messagePattern":"failed query COMMIT, err: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"go/vt/binlog/binlogplayer/binlog_player.go","lineNumber":492,"sourceCode":"\t\tif sqlErr, ok := err.(*sqlerror.SQLError); ok && sqlErr.Number() == sqlerror.ERLockDeadlock {\n\t\t\t// Deadlock: ask for retry\n\t\t\tlog.Info(fmt.Sprintf(\"Deadlock: %v\", err))\n\t\t\tif err = blp.dbClient.Rollback(); err != nil {\n\t\t\t\treturn false, err\n\t\t\t}\n\t\t\treturn false, nil\n\t\t}\n\t\t_ = blp.dbClient.Rollback()\n\t\treturn false, err\n\t}\n\t// Update recovery position after successful replay.\n\t// This also updates the blp's internal position.\n\tif err = blp.writeRecoveryPosition(tx); err != nil {\n\t\t_ = blp.dbClient.Rollback()\n\t\treturn false, err\n\t}\n\tif err = blp.dbClient.Commit(); err != nil {\n\t\treturn false, fmt.Errorf(\"failed query COMMIT, err: %s\", err)\n\t}\n\tblp.blplStats.Timings.Record(BlplTransaction, txnStartTime)\n\treturn true, nil\n}\n\nfunc (blp *BinlogPlayer) exec(sql string) (*sqltypes.Result, error) {\n\tqueryStartTime := time.Now()\n\tqr, err := blp.dbClient.ExecuteFetch(sql, 0)\n\tblp.blplStats.Timings.Record(BlplQuery, queryStartTime)\n\tif d := time.Since(queryStartTime); d > SlowQueryThreshold {\n\t\tlog.Info(fmt.Sprintf(\"SLOW QUERY (took %.2fs) '%s'\", d.Seconds(), sql))\n\t}\n\treturn qr, err\n}\n\n// writeRecoveryPosition writes the current GTID as the recovery position\n// for the next transaction.\n// It also tries to get the timestamp for the transaction. Two cases:","sourceCodeStart":474,"sourceCodeEnd":510,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/vt/binlog/binlogplayer/binlog_player.go#L474-L510","documentation":"Thrown by processTransaction when blp.dbClient.Commit() fails after all statements and the recovery-position write succeeded within the transaction. Because recovery info is written inside the same txn, a failed COMMIT means nothing (including the position) was persisted, so the whole transaction is retried from the stream.","triggerScenarios":"dbClient.Commit() errors: connection dropped between last statement and COMMIT, MySQL killed the transaction (lock wait timeout, deadlock victim), target went read-only, disk full on target, or server crash during commit.","commonSituations":"Long transactions hitting innodb_lock_wait_timeout while other writers hold row locks; disk-full on the target host; network flap exactly at commit time; MySQL crash mid-commit causing recovery rollback.","solutions":["Check the inner error: lock timeout → reduce competing writers or raise innodb_lock_wait_timeout; disk full → free space.","Verify the last committed position in _vt.vreplication; nothing was committed for this txn, so a retry is safe.","Restart the vreplication stream; it re-dials and re-applies from the stored position.","Inspect target MySQL error log for deadlock/aborted transaction entries at the failure time."],"exampleFix":"// before: lock wait timeout during commit\n//   SET GLOBAL innodb_lock_wait_timeout = 60; -- or\n// after: remove competing writer on the target tables\n//   (quiesce manual writes to replicated tables during vreplication)","handlingStrategy":"retry","validationCode":"// pre-flight: ensure target is writable and has disk headroom\nvar readOnly bool\ntargetDB.QueryRow(\"SELECT @@read_only\").Scan(&readOnly)\nif readOnly {\n    return errors.New(\"target read-only; commits will fail\")\n}\n// monitor df / MySQL free-space metrics on the target host","typeGuard":null,"tryCatchPattern":"err := binlogplayer.ApplyBinlogEvents(ctx, blp)\nif err != nil && strings.Contains(err.Error(), \"failed query COMMIT\") {\n    // recovery info is in the same txn: nothing committed; retry is safe\n    log.Warn(\"COMMIT failed; transaction fully rolled back — retrying from stored position\")\n    return retryWithBackoff(ctx, func() error {\n        return binlogplayer.ApplyBinlogEvents(ctx, blp)\n    })\n}","preventionTips":["Quiesce external writers on replicated target tables to avoid lock timeouts","Raise innodb_lock_wait_timeout if long replay transactions contend","Monitor target disk space (commits fail when the disk fills)","Check for deadlock victims in the target MySQL error log"],"tags":["mysql","commit","transaction","binlog-player"],"backgroundTag":"mysql-commit-failed","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}