vitessio/vitess · error
error in processing binlog event
Error message
error in processing binlog event %v
What it means
Thrown by applyEvents when blp.processTransaction fails to apply a received binlog transaction to the target. The failed transaction's statements are logged for diagnosis and the error bubbles up so the stream restarts from the last committed recovery position.
Solutions
- Read the wrapped inner error and the logged `statement:` lines to see which SQL failed and why.
- Fix the target schema to match source (apply migration that diverged).
- Resolve data conflicts: for duplicate-key errors, either delete the conflicting row or verify the recovery position and let the player reapply.
- Check that no other writer is mutating the replicated tables on the target during the stream.
- Restart the stream after fixing; it resumes from the last written recovery position.
Example fix
// before: target missing column, statement fails // ALTER TABLE t ADD COLUMN c VARCHAR(10); -- run on target // after: schema matches, replay proceeds // (stream restarts and processTransaction succeeds)
Defensive patterns
Strategy: try-catch
Validate before calling
// pre-flight: confirm target schema matches source for replicated tables
for _, t := range tables {
src := showCreateTable(sourceDB, t)
tgt := showCreateTable(targetDB, t)
if normalize(src) != normalize(tgt) {
return fmt.Errorf("schema drift on %s: fix before starting stream", t)
}
} Try / catch
// capture failed SQL from logs and the inner error for triage
err := binlogplayer.ApplyBinlogEvents(ctx, blp)
if err != nil && strings.Contains(err.Error(), "error in processing binlog event") {
log.Errorf("replay failed: %v — check 'statement:' log lines for the failing SQL", err)
// fix schema/data, then restart stream; it resumes from recovery position
} Prevention
- Keep target schema in lockstep with source migrations during resharding
- Ensure no external writers touch replicated tables on the target
- Verify MySQL sql_mode and collations match between source and target
- Run schema validation (vtctldclient schema compare) before starting workflows
When it happens
Trigger: processTransaction returns err: BEGIN fails, SET charset fails, statement exec fails (e.g. duplicate key, missing table, SQL mode difference, data type mismatch), recovery-position write fails, or COMMIT fails.
Common situations: Target schema out of sync with source (missing column/table), duplicate rows causing ER_DUP_ENTRY after a partial replay, MySQL sql_mode/collation differences, or a target under external writes conflicting with replayed rows.
Related errors
- can't get charset to request binlog stream
- can't set charset for statement
- checkpoint information not available in db for
- could not get the original sql_mode on target
- could not set message
AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01).
Data as JSON: /api/errors/12bd2138c206c650.
Report an issue: GitHub.
Appendix: source
Thrown at go/vt/binlog/binlogplayer/binlog_player.go:421
// contexts could be wrapped as regular errors.
select {
case <-ctx.Done():
return nil
default:
}
if err != nil {
return fmt.Errorf("error received from Stream %v", err)
}
// process the transaction
for {
ok, err = blp.processTransaction(response)
if err != nil {
log.Info(fmt.Sprintf("transaction failed: %v", err))
for _, stmt := range response.Statements {
log.Info(fmt.Sprintf("statement: %q", stmt.Sql))
}
return fmt.Errorf("error in processing binlog event %v", err)
}
if ok {
if !blp.stopPosition.IsZero() {
if blp.position.AtLeast(blp.stopPosition) {
msg := "Reached stopping position, done playing logs"
log.Info(msg)
if err := blp.setVReplicationState(binlogdatapb.VReplicationWorkflowState_Stopped, msg); err != nil {
log.Error(fmt.Sprintf("Error writing stop state: %v", err))
}
return nil
}
}
break
}
log.Info(fmt.Sprintf("Retrying txn in %v.", blp.deadlockRetry))
time.Sleep(blp.deadlockRetry)
}
}View on GitHub (pinned to 01a25a7d17)