vitessio/vitess · error

failed to lock %s/%s; if you really want to delete this shar

Error message

failed to lock %s/%s; if you really want to delete this shard, re-run with Force=true: %w

What it means

deleteShard tries to take a shard-level topo lock (LockShard with action DeleteShard) before deleting. If locking fails and force is false, deletion is refused with "failed to lock %s/%s; if you really want to delete this shard, re-run with Force=true: %w", protecting against concurrent operations on the shard.

Source

Thrown at go/vt/vtctl/grpcvtctldserver/topo.go:51

)

func deleteShard(ctx context.Context, ts *topo.Server, keyspace string, shard string, recursive bool, evenIfServing bool, force bool) (err error) {
	span, ctx := trace.NewSpan(ctx, "VtctldServer.deleteShard")
	defer span.Finish()

	span.Annotate("keyspace", keyspace)
	span.Annotate("shard", shard)
	span.Annotate("recursive", recursive)
	span.Annotate("even_if_serving", evenIfServing)
	span.Annotate("force", force)

	lctx, unlock, lerr := ts.LockShard(ctx, keyspace, shard, "DeleteShard")
	switch {
	case lerr == nil:
		// We locked the shard, all good
		ctx = lctx
	case !force:
		return fmt.Errorf("failed to lock %s/%s; if you really want to delete this shard, re-run with Force=true: %w", keyspace, shard, lerr)
	default:
		// Failed to lock, but force=true. Warn and continue
		log.Warn(fmt.Sprintf("%s: failed to lock shard %s/%s for deletion, but force=true, proceeding anyway ...", lerr, keyspace, shard))
	}

	if unlock != nil {
		defer func() {
			// Attempting to unlock a shard we successfully deleted results in
			// ts.unlockShard returning an error, which can make the overall
			// RPC _seem_ like it failed.
			//
			// So, we do this extra checking to allow for specifically this
			// scenario to result in "success."
			origErr := err
			unlock(&err)
			if origErr == nil && topo.IsErrType(err, topo.NoNode) {
				err = nil
			}

View on GitHub (pinned to 01a25a7d17)

Solutions

  1. Wait for the concurrent operation to finish and its lock to be released, then re-run without Force.
  2. Verify nothing is actually operating on the shard, then re-run with Force=true to bypass the lock.
  3. Inspect the topo lock node (GetShardLockInfo / topo lock paths) to identify the lock holder and clear a stale lock.

Example fix

// before
vtctldclient DeleteShard commerce/0
// error: failed to lock commerce/0 ...
// after (only if safe — no active operations on the shard)
vtctldclient DeleteShard commerce/0 --force
Defensive patterns

Strategy: try-catch

Validate before calling

lctx, unlock, err := ts.LockShard(ctx, ks, shard, "check-only"); if err != nil { /* lock is held: find holder via topo lock info before destructive ops */ } else { unlock(&err) }

Try / catch

err := deleteShard(ctx, ks, shard, false); if err != nil && strings.Contains(err.Error(), "failed to lock") { /* confirm no active op on the shard, then retry with Force=true or wait for lock expiry */ }

Prevention

When it happens

Trigger: DeleteShard/DeleteKeyspace while another operation holds the shard lock (stale lock from a crashed job, or a live PRS/backup), and Force is not set.

Common situations: Aborted vtctl run left a lock entry in the topo; another operator or workflow is actively working on the shard; lock TTL not yet expired.

Related errors


AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/13121ee7d96d2a69. Report an issue: GitHub.