vitessio/vitess · error

stream %d has not started on tablet %v

Error message

stream %d has not started on tablet %v

What it means

While stopping target vreplication streams during table_differ initialization, the differ reads each stream's recorded position. A zero (empty) binlog position means the stream was created but never started replicating, so there is nothing valid to stop/park — the differ returns this error.

Source

Thrown at go/vt/vttablet/tabletmanager/vdiff/table_differ.go:227

		return err
	}
	// streams are no longer running because vre.Exec would have replaced old controllers and new ones will not start

	// update position of all source streams
	query = "select id, source, pos from _vt.vreplication " + ct.workflowFilter
	qr, err := dbClient.ExecuteFetch(query, -1)
	if err != nil {
		return err
	}
	for _, row := range qr.Named().Rows {
		id, _ := row["id"].ToInt64()
		pos := row["pos"].ToString()
		mpos, err := binlogplayer.DecodePosition(pos)
		if err != nil {
			return err
		}
		if mpos.IsZero() {
			return fmt.Errorf("stream %d has not started on tablet %v",
				id, td.wd.ct.vde.thisTablet.Alias)
		}
		sourceBytes, err := row["source"].ToBytes()
		if err != nil {
			return err
		}
		var bls binlogdatapb.BinlogSource
		if err := prototext.Unmarshal(sourceBytes, &bls); err != nil {
			return err
		}
		ct.sources[bls.Shard].position = mpos
	}

	return nil
}

func (td *tableDiffer) forEachSource(cb func(source *migrationSource) error) error {
	ct := td.wd.ct

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Check the target stream's state/message: SELECT id, state, message, pos FROM _vt.vreplication; on the target tablet.
2. Fix the underlying stream failure (source tablet reachability, permissions, filter) and let it start before running vdiff.
3. If the stream is dead, delete/recreate the workflow (MoveTables/Reshard) so streams start properly, then re-run vdiff.

Example fix

// before
row["pos"] = "" // stream never started
// after (verify first)
SELECT id, pos, state FROM _vt.vreplication WHERE id=<stream id>;
-- pos must be a non-zero gtid set before vdiff

rows := queryTarget("SELECT id, pos, state FROM _vt.vreplication WHERE workflow=?", wf)
for _, r := range rows {
  if r["state"].ToString() != "Running" || r["pos"].ToString() == "" {
    return errors.New("wait for streams to start before vdiff")
  }
}

if err := startVDiff(); err != nil && strings.Contains(err.Error(), "has not started") {
  // poll until stream pos is non-zero, then retry
  waitForStreamStarted(streamID)
  retryVDiff()
}

When it happens

Trigger: initialize → stopTargetVReplicationStreams finds a target _vt.vreplication row whose pos column decodes to a zero position, i.e. the stream never made progress after creation.

Common situations: Target vreplication stream stuck/failed at startup (source unreachable, wrong keyspace/shard); racing a vdiff create immediately after workflow creation before streams start; target tablet copy loop stalled.

Related errors

• error received from Stream %v
• can only show a specific vdiff, please provide a valid UUID;
• can only %s a specific vdiff, please provide a valid UUID; v
• can only delete a specific vdiff, please provide a valid UUI
• invalid action %s; %s

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/de2e1429fd3671e6. Report an issue: GitHub.