vitessio/vitess · error
failed to lock %s/%s; if you really want to delete this shar
Error message
failed to lock %s/%s; if you really want to delete this shard, re-run with Force=true: %w
What it means
deleteShard tries to take a shard-level topo lock (LockShard with action DeleteShard) before deleting. If locking fails and force is false, deletion is refused with "failed to lock %s/%s; if you really want to delete this shard, re-run with Force=true: %w", protecting against concurrent operations on the shard.
Source
Thrown at go/vt/vtctl/grpcvtctldserver/topo.go:51
)
func deleteShard(ctx context.Context, ts *topo.Server, keyspace string, shard string, recursive bool, evenIfServing bool, force bool) (err error) {
span, ctx := trace.NewSpan(ctx, "VtctldServer.deleteShard")
defer span.Finish()
span.Annotate("keyspace", keyspace)
span.Annotate("shard", shard)
span.Annotate("recursive", recursive)
span.Annotate("even_if_serving", evenIfServing)
span.Annotate("force", force)
lctx, unlock, lerr := ts.LockShard(ctx, keyspace, shard, "DeleteShard")
switch {
case lerr == nil:
// We locked the shard, all good
ctx = lctx
case !force:
return fmt.Errorf("failed to lock %s/%s; if you really want to delete this shard, re-run with Force=true: %w", keyspace, shard, lerr)
default:
// Failed to lock, but force=true. Warn and continue
log.Warn(fmt.Sprintf("%s: failed to lock shard %s/%s for deletion, but force=true, proceeding anyway ...", lerr, keyspace, shard))
}
if unlock != nil {
defer func() {
// Attempting to unlock a shard we successfully deleted results in
// ts.unlockShard returning an error, which can make the overall
// RPC _seem_ like it failed.
//
// So, we do this extra checking to allow for specifically this
// scenario to result in "success."
origErr := err
unlock(&err)
if origErr == nil && topo.IsErrType(err, topo.NoNode) {
err = nil
}View on GitHub (pinned to 01a25a7d17)
Solutions
- Wait for the concurrent operation to finish and its lock to be released, then re-run without Force.
- Verify nothing is actually operating on the shard, then re-run with Force=true to bypass the lock.
- Inspect the topo lock node (GetShardLockInfo / topo lock paths) to identify the lock holder and clear a stale lock.
Example fix
// before vtctldclient DeleteShard commerce/0 // error: failed to lock commerce/0 ... // after (only if safe — no active operations on the shard) vtctldclient DeleteShard commerce/0 --force
Defensive patterns
Strategy: try-catch
Validate before calling
lctx, unlock, err := ts.LockShard(ctx, ks, shard, "check-only"); if err != nil { /* lock is held: find holder via topo lock info before destructive ops */ } else { unlock(&err) } Try / catch
err := deleteShard(ctx, ks, shard, false); if err != nil && strings.Contains(err.Error(), "failed to lock") { /* confirm no active op on the shard, then retry with Force=true or wait for lock expiry */ } Prevention
- Never force-delete while reparents/backups may be running
- Check shard lock info before destructive commands
- Use short-lived operations so stale locks expire quickly
- Only pass Force=true after verifying the lock holder is dead
When it happens
Trigger: DeleteShard/DeleteKeyspace while another operation holds the shard lock (stale lock from a crashed job, or a live PRS/backup), and Force is not set.
Common situations: Aborted vtctl run left a lock entry in the topo; another operator or workflow is actively working on the shard; lock TTL not yet expired.
Related errors
- shard %v/%v has no primary
- can't get primary tablet record %v: %v
- cannot get (or create) shard %v/%v: %v
- shard %v/%v has a different KeyRange: %v != %v
- old tablet has shard %v/%v. Cannot override with shard %v/%v
AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01).
Data as JSON: /api/errors/13121ee7d96d2a69.
Report an issue: GitHub.