vitessio/vitess · error

cannot migrate while vreplication streams in source shards a

Error message

cannot migrate while vreplication streams in source shards are still copying: %s

What it means

After collecting all tablet streams on a source shard, the migrator checks _vt.copy_state for rows belonging to those streams. If any stream still has copy-state entries it is still in the bulk-copy phase and its position is not stable; migrating a copying stream would lose data, so the migration aborts naming the source shard.

Source

Thrown at go/vt/vtctl/workflow/stream_migrator.go:480

		}

		tabletStreams, err := sm.legacyReadTabletStreams(ctx, source.GetPrimary(), "")
		if err != nil {
			return err
		}

		if len(tabletStreams) == 0 {
			// No VReplication is running. So, we have no work to do.
			return nil
		}

		query := "select distinct vrepl_id from _vt.copy_state where vrepl_id in " + VReplicationStreams(tabletStreams).Values()
		p3qr, err := sm.ts.TabletManagerClient().VReplicationExec(ctx, source.GetPrimary().Tablet, query)
		switch {
		case err != nil:
			return err
		case len(p3qr.Rows) != 0:
			return fmt.Errorf("cannot migrate while vreplication streams in source shards are still copying: %s", source.GetShard().ShardName())
		}

		mu.Lock()
		defer mu.Unlock()
		streams[source.GetShard().ShardName()] = tabletStreams
		return nil
	})
	if err != nil {
		return nil, err
	}

	// Validate that streams match across source shards.
	var (
		reference []*VReplicationStream
		refshard  string
		streams2  = make(map[string][]*VReplicationStream)
	)

View on GitHub (pinned to 01a25a7d17)

Solutions

  1. Wait until all streams finish copying: monitor `select count(*) from _vt.copy_state` / vtctldclient Workflow status until copy phase completes.
  2. If a copy is stalled or unwanted, remove that stream and restart it before migrating.
  3. Throttle/backfill large tables or use faster copy settings so the copy phase finishes, then retry the migration.
  4. Only run stream migration once every source-shard stream reports Running (not Copying) with empty copy_state.

Example fix

// before: migrate attempted while stream is copying
$ vtctldclient Workflow --keyspace customer Migrate ...
ERROR: cannot migrate while vreplication streams in source shards are still copying: -40
// after: wait for copy to finish
$ watch 'mysql -S ... -e "select count(*) from _vt.copy_state"' # wait for 0
$ vtctldclient Workflow --keyspace customer Migrate ...
Defensive patterns

Strategy: validation

Validate before calling

qr, _ := tmclient.VReplicationExec(ctx, tablet, "select distinct vrepl_id from _vt.copy_state")
if len(qr.Rows) != 0 {
    return fmt.Errorf("wait for copy phase on %d stream(s) before migrating", len(qr.Rows))
}

Try / catch

err := migrateStreams(ctx, cfg)
if err != nil && strings.Contains(err.Error(), "still copying") {
    // poll _vt.copy_state until empty on the named shard, then retry
}

Prevention

When it happens

Trigger: Calling MigrateStreams / workflow stream migration while a stream on the source shard is actively copying tables — i.e. `select distinct vrepl_id from _vt.copy_state where vrepl_id in (...)` returns rows for any of the shard's streams.

Common situations: Newly created streams still bulk-copying large tables; copy phase stalled or very slow due to large tables/network; operator attempted migration too soon after starting streams.

Related errors


AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/f95c0f49d55458a3. Report an issue: GitHub.