vitessio/vitess · error · ErrMultipleSourceKeyspaces

multiple source keyspaces for a single workflow

Error message

multiple source keyspaces for a single workflow

What it means

ErrMultipleSourceKeyspaces is returned by scanWorkflow when the vreplication streams of a single workflow report more than one source keyspace across different shard primaries. This should be impossible for a healthy workflow, so it signals inconsistent or corrupted workflow metadata.

Source

Thrown at go/vt/vtctl/workflow/server.go:122

	cannotSwitchFailedTabletRefresh = "could not refresh all of the tablets involved in the operation:\n%s"
	cannotSwitchFrozen              = "workflow is frozen"

	// Number of LOCK TABLES cycles to perform on the sources during SwitchWrites.
	lockTablesCycles = 2
	// Time to wait between LOCK TABLES cycles on the sources during SwitchWrites.
	lockTablesCycleDelay = time.Duration(100 * time.Millisecond)

	SqlUnfreezeWorkflow = "update _vt.vreplication set state='Running', message='' where db_name=%a and workflow=%a"
)

var (
	// ErrInvalidWorkflow is a catchall error type for conditions that should be
	// impossible when operating on a workflow.
	ErrInvalidWorkflow = errors.New("invalid workflow")
	// ErrMultipleSourceKeyspaces occurs when a workflow somehow has multiple
	// source keyspaces across different shard primaries. This should be
	// impossible.
	ErrMultipleSourceKeyspaces = errors.New("multiple source keyspaces for a single workflow")
	// ErrMultipleTargetKeyspaces occurs when a workflow somehow has multiple
	// target keyspaces across different shard primaries. This should be
	// impossible.
	ErrMultipleTargetKeyspaces          = errors.New("multiple target keyspaces for a single workflow")
	ErrWorkflowCompleteNotFullySwitched = errors.New("cannot complete workflow because you have not yet switched all read and write traffic")
	ErrWorkflowDeleteWritesSwitched     = errors.New("cannot delete workflow because you have already switched write traffic")
)

// Server provides an API to work with Vitess workflows, like vreplication
// workflows (MoveTables, Reshard, etc) and schema migration workflows.
type Server struct {
	ts  *topo.Server
	tmc tmclient.TabletManagerClient
	// Limit the number of concurrent background goroutines if needed.
	sem     *semaphore.Weighted
	env     *vtenv.Environment
	options serverOptions
}

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Inspect `select workflow, source from _vt.vreplication` on all shards to find the divergent rows.
2. Delete or fix the offending stream rows so all streams share one source keyspace.
3. Recreate the workflow if the metadata cannot be reconciled.

Example fix

// before
rows with source keyspaces: commerce, commerce2 -> scanWorkflow returns ErrMultipleSourceKeyspaces
// after
mysql> delete from _vt.vreplication where id=<offending id>;
// all streams now report source keyspace 'commerce'

sources := distinctSourceKeyspaces(keyspace, workflow)
if len(sources) > 1 { return workflow.ErrMultipleSourceKeyspaces }

func isMultipleSourceKeyspaces(err error) bool {
    return errors.Is(err, workflow.ErrMultipleSourceKeyspaces)
}

if err := scanWorkflow(ctx, ks, wf); err != nil {
    if errors.Is(err, workflow.ErrMultipleSourceKeyspaces) {
        // repair or delete the divergent stream rows
    }
    return err
}

When it happens

Trigger: Calling any server API that scans a workflow (scanWorkflow) where rows in _vt.vreplication across the source shards resolve to different source keyspaces.

Common situations: Manual edits to _vt.vreplication, leftover rows from a merged or misconfigured Reshard, or streams created by different tool versions writing inconsistent specs.

Related errors

• multiple target keyspaces for a single workflow
• both atomic copy and partial mode cannot be specified for th
• invalid workflow
• value out of range
• no streams found

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/aca89c6bb3d43053. Report an issue: GitHub.