vitessio/vitess · error

error in writing recovery info

Error message

error %v in writing recovery info %v

What it means

Thrown by writeRecoveryPosition when the UPDATE of the _vt.vreplication row (GenerateUpdatePos — storing new position, timestamp, row counts) fails to execute. The caller rolls back the current transaction so no data and no position advance is persisted, keeping replay consistent.

Solutions

1. Verify _vt.vreplication exists and is intact on the target (`SELECT id, pos FROM _vt.vreplication WHERE id=<uid>`).
2. Check the vttablet/vreplication user's grants on _vt.vreplication (UPDATE at minimum).
3. Ensure target is writable (read_only=0) and reachable.
4. Examine the updateRecovery SQL in the error message for malformed position data; report/fix if GeneratePos output is corrupt.

Example fix

// before: grants revoked
//   GRANT SELECT, INSERT ON _vt.vreplication TO 'vt'@'%';
// after: include UPDATE for recovery writes
//   GRANT SELECT, INSERT, UPDATE ON _vt.vreplication TO 'vt'@'%';

// before streaming, verify the recovery row exists and is writable
var n int
if err := targetDB.QueryRow(
    "SELECT COUNT(*) FROM _vt.vreplication WHERE id = ?", uid,
).Scan(&n); err != nil || n != 1 {
    return fmt.Errorf("_vt.vreplication row id=%d missing or table unreadable", uid)
}
// also verify UPDATE grant:
// SHOW GRANTS FOR CURRENT_USER() — must include UPDATE ON _vt.*

When it happens

Trigger: blp.exec(updateRecovery) returns err: target connection lost, SQL error in the generated UPDATE (e.g. malformed position string, table _vt.vreplication missing), permissions revoked on _vt.vreplication, or target read-only.

Common situations: Someone dropped/altered _vt.vreplication during a migration; vttablet user lost INSERT/UPDATE grants on _vt; target in read_only during a maintenance window; connection expired between COMMIT of previous txn and this write.

Related errors

• can't get charset to request binlog stream
• cannot update vreplication table, affected
• checkpoint information not available in db for
• could not get the original sql_mode on target
• could not set message

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/4fba5413ec0d85be. Report an issue: GitHub.

Appendix: source

Thrown at go/vt/binlog/binlogplayer/binlog_player.go:528

// It also tries to get the timestamp for the transaction. Two cases:
//   - we have statements, and they start with a SET TIMESTAMP that we
//     can parse: then we update transaction_timestamp in vreplication
//     with it, and set ReplicationLagSeconds to now() - transaction_timestamp
//   - otherwise (the statements are probably filtered out), we leave
//     transaction_timestamp alone (keeping the old value), and we don't
//     change ReplicationLagSeconds
func (blp *BinlogPlayer) writeRecoveryPosition(tx *binlogdatapb.BinlogTransaction) error {
	position, err := DecodePosition(tx.EventToken.Position)
	if err != nil {
		return err
	}

	now := time.Now().Unix()
	updateRecovery := GenerateUpdatePos(blp.uid, position, now, tx.EventToken.Timestamp, blp.blplStats.CopyRowCount.Get(), false)

	qr, err := blp.exec(updateRecovery)
	if err != nil {
		return fmt.Errorf("error %v in writing recovery info %v", err, updateRecovery)
	}
	if qr.RowsAffected != 1 {
		return fmt.Errorf("cannot update vreplication table, affected %v rows", qr.RowsAffected)
	}

	// Update position after successful write.
	blp.position = position
	blp.blplStats.SetLastPosition(blp.position)
	if tx.EventToken.Timestamp != 0 {
		blp.blplStats.ReplicationLagSeconds.Store(now - tx.EventToken.Timestamp)
	}
	return nil
}

func (blp *BinlogPlayer) setVReplicationState(state binlogdatapb.VReplicationWorkflowState, message string) error {
	if message != "" {
		blp.blplStats.History.Add(&StatsHistoryRecord{
			Time:    time.Now(),

View on GitHub (pinned to 01a25a7d17)