vitessio/vitess · error

missing source

Error message

missing source

What it means

The vreplication controller (runBlp) reaches a state where its source tablet/position is unset and returns 'missing source' after incrementing the 'Invalid Source' error counter. runBlp needs a valid source (tablet + binlog position) to start replicating; if the workflow's source record is missing it cannot continue.

Source

Thrown at go/vt/vttablet/tabletmanager/vreplication/controller.go:361

		// it's a FAILED_PRECONDITION vterror, OR we cannot identify this as
		// non-recoverable BUT it has persisted beyond the retry limit
		// (maxTimeToRetryError). In addition, we cannot restart a workflow
		// started with AtomicCopy which has _any_ error during copy phase.
		if (err != nil && vr.WorkflowSubType == int32(binlogdatapb.VReplicationWorkflowSubType_AtomicCopy) && vr.state == binlogdatapb.VReplicationWorkflowState_Copying) ||
			isUnrecoverableError(err) ||
			!ct.lastWorkflowError.ShouldRetry() {
			err = vterrors.Wrapf(err, TerminalErrorIndicator)
			if errSetState := vr.setState(binlogdatapb.VReplicationWorkflowState_Error, err.Error()); errSetState != nil {
				log.Error(fmt.Sprintf("INTERNAL: unable to setState() in controller: %v. Could not set error text to: %v.", errSetState, err))
				return err // yes, err and not errSetState.
			}
			log.Error(fmt.Sprintf("%s going into error state due to %+v", ct.logPrefix(), err))
			return nil // this will cause vreplicate to quit the workflow
		}
		return err
	}
	ct.blpStats.ErrorCounts.Add([]string{"Invalid Source"}, 1)
	return errors.New("missing source")
}

func (ct *controller) setMessage(dbClient binlogplayer.DBClient, message string) error {
	ct.blpStats.History.Add(&binlogplayer.StatsHistoryRecord{
		Time:    time.Now(),
		Message: message,
	})
	query := fmt.Sprintf("update _vt.vreplication set message=%v where id=%v", encodeString(binlogplayer.MessageTruncate(message)), ct.id)
	if _, err := dbClient.ExecuteFetch(query, 1); err != nil {
		return fmt.Errorf("could not set message: %v: %v", query, err)
	}
	return nil
}

// pickSourceTablet picks a healthy serving tablet to source for
// the vreplication stream. If the source is marked as external, it
// returns nil.
func (ct *controller) pickSourceTablet(ctx context.Context, dbClient binlogplayer.DBClient) (*topodatapb.Tablet, error) {

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Re-check the workflow source with vtctl VReplicationExec / Workflow show and confirm the source tablet and keyspace/shard are valid
2. Recreate or restart the workflow (MoveTables/Reshard create) so the source is re-resolved against the current topo
3. If the source was intentionally removed, cancel the stale _vt.vreplication row (vtctl Workflow ... cancel/delete)
4. Check source tablet health/registration in the topo and re-run the workflow

Example fix

// before
$ vtctlclient VReplicationExec <tablet> 'show'  // source missing
// after
$ vtctlclient MoveTables -workflow=sales -target ks2 create ... // recreate stream with valid source

// verify the workflow source resolves before starting
rows, _ := db.ExecuteFetch("select source, pos from _vt.vreplication", 100)

if err := runBlp(ctx); err != nil && strings.Contains(err.Error(), "missing source") {
    // recreate/cancel the stale workflow row
    vtctldclient Workflow --keyspace ks cancel <workflow>
}

When it happens

Trigger: Starting/running a vreplication stream whose _vt.vreplication row's source (tablet alias / source keyspace-shard) cannot be resolved, so blpFunc signals the source is invalid.

Common situations: Source tablet decommissioned or moved without updating the workflow; pruned/shard-merged workflows with stale source entries; manual edits to _vt.vreplication; failed MoveTables/Reshard source resolution.

Related errors

• value out of range
• both atomic copy and partial mode cannot be specified for th
• invalid workflow
• multiple source keyspaces for a single workflow
• multiple target keyspaces for a single workflow

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/40d8cfa4caca7eff. Report an issue: GitHub.