vitessio/vitess · error

could not set message: %v: %v

Error message

could not set message: %v: %v

What it means

setMessage records a human-readable status message for a vreplication stream by running `update _vt.vreplication set message=... where id=...` through the controller's dbClient. If that UPDATE fails, the query and underlying MySQL error are wrapped as `could not set message: <query>: <err>`.

Source

Thrown at go/vt/vttablet/tabletmanager/vreplication/controller.go:371

				return err // yes, err and not errSetState.
			}
			log.Error(fmt.Sprintf("%s going into error state due to %+v", ct.logPrefix(), err))
			return nil // this will cause vreplicate to quit the workflow
		}
		return err
	}
	ct.blpStats.ErrorCounts.Add([]string{"Invalid Source"}, 1)
	return errors.New("missing source")
}

func (ct *controller) setMessage(dbClient binlogplayer.DBClient, message string) error {
	ct.blpStats.History.Add(&binlogplayer.StatsHistoryRecord{
		Time:    time.Now(),
		Message: message,
	})
	query := fmt.Sprintf("update _vt.vreplication set message=%v where id=%v", encodeString(binlogplayer.MessageTruncate(message)), ct.id)
	if _, err := dbClient.ExecuteFetch(query, 1); err != nil {
		return fmt.Errorf("could not set message: %v: %v", query, err)
	}
	return nil
}

// pickSourceTablet picks a healthy serving tablet to source for
// the vreplication stream. If the source is marked as external, it
// returns nil.
func (ct *controller) pickSourceTablet(ctx context.Context, dbClient binlogplayer.DBClient) (*topodatapb.Tablet, error) {
	if ct.source.GetExternalMysql() != "" {
		return nil, nil
	}
	if ct.tpTs == nil {
		return nil, fmt.Errorf("no tablet picker configured for %s/%s", ct.source.Keyspace, ct.source.Shard)
	}
	log.Info(fmt.Sprintf("%s trying to find an eligible source tablet in %s/%s", ct.logPrefix(), ct.source.Keyspace, ct.source.Shard))

	tablet, err := ct.pickSourceTabletWithIgnoreList(ctx, dbClient)
	// If we failed to find any tablet and we specified an ignore list, then clear the ignore list so that

View on GitHub (pinned to 01a25a7d17)

Solutions

  1. Look at the embedded MySQL error after the second %v — it names the real cause (table missing, connection refused, etc.).
  2. Verify the vreplication row still exists: `select id, message from _vt.vreplication where id=<ct.id>`.
  3. Confirm the sidecar schema is up to date (`_vt.vreplication` table exists and is current).
  4. Check tablet-to-MySQL connectivity and privileges on the _vt database.

Example fix

// before (only generic wrap, hard to diagnose)
return fmt.Errorf("could not set message: %v: %v", query, err)
// after — diagnose by inspecting the wrapped cause
err := fmt.Errorf("could not set message: %v: %v", query, err)
// then run on the tablet's MySQL:
// select id from _vt.vreplication where id = <stream-id>;
Defensive patterns

Strategy: try-catch

Validate before calling

// Before workflows, verify the stream row and table exist:
// select id, workflow, message from _vt.vreplication where id = <stream-id>;
// show tables in _vt like 'vreplication';

Try / catch

if _, err := dbClient.ExecuteFetch(q, 1); err != nil {
    if mysqlErr, ok := err.(*sqlexec.UnknownDatabaseError); ok { /* recreate sidecar */ }
    return vterrors.Wrapf(err, "could not set message for stream %d", streamID)
}

Prevention

When it happens

Trigger: The `update _vt.vreplication set message=...` statement fails — e.g. the sidecar database or _vt.vreplication table is missing, the row for ct.id was deleted (stream stopped), or the MySQL connection used by dbClient is broken.

Common situations: vreplication row deleted concurrently by StopVCopy/UpdateVReplicationWorkflow while the controller is still running; sidecar schema not migrated; MySQL restart or failover dropping the connection; permissions revoked on the _vt database.

Related errors

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/a8e2ecba5e8e846b. Report an issue: GitHub.