vitessio/vitess · error

target shard %v is in serving state

Error message

target shard %v is in serving state

What it means

Mirror of the source-shard check: reshard requires target shards to NOT be primary-serving, because a serving target would take live traffic and accept writes while VReplication is still copying data. If any target shard has IsPrimaryServing=true, Reshard fails with this error.

Source

Thrown at go/vt/wrangler/resharder.go:139

			return nil, vterrors.Wrapf(err, "GetShard(%s) failed", shard)
		}
		if !si.IsPrimaryServing {
			return nil, fmt.Errorf("source shard %v is not in serving state", shard)
		}
		rs.sourceShards = append(rs.sourceShards, si)
		primary, err := wr.ts.GetTablet(ctx, si.PrimaryAlias)
		if err != nil {
			return nil, vterrors.Wrapf(err, "GetTablet(%s) failed", si.PrimaryAlias)
		}
		rs.sourcePrimaries[si.ShardName()] = primary
	}
	for _, shard := range targets {
		si, err := wr.ts.GetShard(ctx, keyspace, shard)
		if err != nil {
			return nil, vterrors.Wrapf(err, "GetShard(%s) failed", shard)
		}
		if si.IsPrimaryServing {
			return nil, fmt.Errorf("target shard %v is in serving state", shard)
		}
		rs.targetShards = append(rs.targetShards, si)
		primary, err := wr.ts.GetTablet(ctx, si.PrimaryAlias)
		if err != nil {
			return nil, vterrors.Wrapf(err, "GetTablet(%s) failed", si.PrimaryAlias)
		}
		rs.targetPrimaries[si.ShardName()] = primary
	}
	if err := topotools.ValidateForReshard(rs.sourceShards, rs.targetShards); err != nil {
		return nil, vterrors.Wrap(err, "ValidateForReshard")
	}
	if err := rs.validateTargets(ctx); err != nil {
		return nil, vterrors.Wrap(err, "validateTargets")
	}

	vschema, err := wr.ts.GetVSchema(ctx, keyspace)
	if err != nil {
		return nil, vterrors.Wrap(err, "GetVSchema")

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Identify the offending target shard(s) from the error and check `vtctlclient GetShard <keyspace>/<shard>`.
2. Disable serving: `vtctlclient SetShardIsPrimaryServing <keyspace>/<target_shard> false`.
3. If the target shards hold stale state from a failed prior reshard, consider cleaning them (delete tablets/shard records) and recreating before re-running Reshard.
4. Re-run the Reshard command after all targets report serving=false.

Example fix

// before
$ vtctlclient GetShard commerce/-80  # IsPrimaryServing: true
// after
$ vtctlclient SetShardIsPrimaryServing commerce/-80 false
$ vtctlclient Reshard commerce.reshard - -80,80-

for _, shard := range targetShards {
    si, err := ts.GetShard(ctx, keyspace, shard)
    if err != nil { return err }
    if si.IsPrimaryServing {
        return fmt.Errorf("pre-flight: target shard %s must not be serving", shard)
    }
}

func targetIsNotServing(si *topo.ShardInfo) bool { return !si.IsPrimaryServing }

err := wr.Reshard(ctx, keyspace, workflow, sources, targets, opts...)
if err != nil && strings.Contains(err.Error(), "is in serving state") {
    // disable serving on target shards, clean stale state, then retry
    return disableTargetServingThenRetry(ctx, keyspace, targets)
}

When it happens

Trigger: Calling Reshard when a target shard (e.g. the new -80 or 80- shards in a split) already has IsPrimaryServing=true — typically because a previous reshard attempt was rolled back incorrectly, or the target shards were pre-created and accidentally enabled for serving.

Common situations: Re-running a reshard into shards left serving from an aborted attempt; someone toggled serving on target shards to test; automated tooling created target shards with serving enabled by default in a custom setup.

Related errors

• source shard %v is not in serving state
• VReplication streams must have named workflows for migration
• streams are mismatched across source shards for workflow: %s
• streams are mismatched across source shards: %v
• cannot reshard streams with a mix of reference and sharded t

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/add54c8cf514a3d9. Report an issue: GitHub.