vitessio/vitess · error

could not set message

Error message

could not set message: %v: %v

What it means

setMessage records a human-readable status message for a vreplication stream by running `update _vt.vreplication set message=... where id=...` through the controller's dbClient. If that UPDATE fails, the query and underlying MySQL error are wrapped as `could not set message: <query>: <err>`.

Solutions

  1. Look at the embedded MySQL error after the second %v — it names the real cause (table missing, connection refused, etc.).
  2. Verify the vreplication row still exists: `select id, message from _vt.vreplication where id=<ct.id>`.
  3. Confirm the sidecar schema is up to date (`_vt.vreplication` table exists and is current).
  4. Check tablet-to-MySQL connectivity and privileges on the _vt database.

Example fix

// before (only generic wrap, hard to diagnose)
return fmt.Errorf("could not set message: %v: %v", query, err)
// after — diagnose by inspecting the wrapped cause
err := fmt.Errorf("could not set message: %v: %v", query, err)
// then run on the tablet's MySQL:
// select id from _vt.vreplication where id = <stream-id>;
Defensive patterns

Strategy: try-catch

Validate before calling

// Before workflows, verify the stream row and table exist:
// select id, workflow, message from _vt.vreplication where id = <stream-id>;
// show tables in _vt like 'vreplication';

Try / catch

if _, err := dbClient.ExecuteFetch(q, 1); err != nil {
    if mysqlErr, ok := err.(*sqlexec.UnknownDatabaseError); ok { /* recreate sidecar */ }
    return vterrors.Wrapf(err, "could not set message for stream %d", streamID)
}

Prevention

When it happens

Trigger: The `update _vt.vreplication set message=...` statement fails — e.g. the sidecar database or _vt.vreplication table is missing, the row for ct.id was deleted (stream stopped), or the MySQL connection used by dbClient is broken.

Common situations: vreplication row deleted concurrently by StopVCopy/UpdateVReplicationWorkflow while the controller is still running; sidecar schema not migrated; MySQL restart or failover dropping the connection; permissions revoked on the _vt database.

Related errors

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/a8e2ecba5e8e846b. Report an issue: GitHub.

Appendix: source

Thrown at go/vt/vttablet/tabletmanager/vreplication/controller.go:371

				return err // yes, err and not errSetState.
			}
			log.Error(fmt.Sprintf("%s going into error state due to %+v", ct.logPrefix(), err))
			return nil // this will cause vreplicate to quit the workflow
		}
		return err
	}
	ct.blpStats.ErrorCounts.Add([]string{"Invalid Source"}, 1)
	return errors.New("missing source")
}

func (ct *controller) setMessage(dbClient binlogplayer.DBClient, message string) error {
	ct.blpStats.History.Add(&binlogplayer.StatsHistoryRecord{
		Time:    time.Now(),
		Message: message,
	})
	query := fmt.Sprintf("update _vt.vreplication set message=%v where id=%v", encodeString(binlogplayer.MessageTruncate(message)), ct.id)
	if _, err := dbClient.ExecuteFetch(query, 1); err != nil {
		return fmt.Errorf("could not set message: %v: %v", query, err)
	}
	return nil
}

// pickSourceTablet picks a healthy serving tablet to source for
// the vreplication stream. If the source is marked as external, it
// returns nil.
func (ct *controller) pickSourceTablet(ctx context.Context, dbClient binlogplayer.DBClient) (*topodatapb.Tablet, error) {
	if ct.source.GetExternalMysql() != "" {
		return nil, nil
	}
	if ct.tpTs == nil {
		return nil, fmt.Errorf("no tablet picker configured for %s/%s", ct.source.Keyspace, ct.source.Shard)
	}
	log.Info(fmt.Sprintf("%s trying to find an eligible source tablet in %s/%s", ct.logPrefix(), ct.source.Keyspace, ct.source.Shard))

	tablet, err := ct.pickSourceTabletWithIgnoreList(ctx, dbClient)
	// If we failed to find any tablet and we specified an ignore list, then clear the ignore list so that

View on GitHub (pinned to 01a25a7d17)