vitessio/vitess · error

can't delete tablet %v: %v

Error message

can't delete tablet %v: %v

What it means

In recursive DeleteShard, each remaining tablet is deleted via wr.DeleteTablet before removing the shard. If any tablet deletion fails (e.g. it is a primary with replicas, or topo update fails), the whole DeleteShard aborts so the replication graph is never removed while a tablet record would become orphaned.

Source

Thrown at go/vt/wrangler/shard.go:150

				return fmt.Errorf("shard %v/%v still has %v tablets in cell %v; use -recursive or remove them manually", keyspace, shard, len(tabletMap), cell)
			}

			wr.Logger().Infof("Deleting all tablets in shard %v/%v cell %v", keyspace, shard, cell)
			for tabletAlias, tabletInfo := range tabletMap {
				// We don't care about scrapping or updating the replication graph,
				// because we're about to delete the entire replication graph.
				wr.Logger().Infof("Deleting tablet %v", tabletAlias)
				if err := wr.TopoServer().DeleteTablet(ctx, tabletInfo.Alias); err != nil && !topo.IsErrType(err, topo.NoNode) {
					// We don't want to continue if a DeleteTablet fails for
					// any good reason (other than missing tablet, in which
					// case it's just a topology server inconsistency we can
					// ignore). If we continue and delete the replication
					// graph, the tablet record will be orphaned, since
					// we'll no longer know it belongs to this shard.
					//
					// If the problem is temporary, or resolved externally, re-running
					// DeleteShard will skip over tablets that were already deleted.
					return fmt.Errorf("can't delete tablet %v: %v", tabletAlias, err)
				}
			}
		}
	}

	// Try to remove the replication graph and serving graph in each cell,
	// regardless of its existence.
	for _, cell := range cells {
		if err := wr.ts.DeleteShardReplication(ctx, cell, keyspace, shard); err != nil && !topo.IsErrType(err, topo.NoNode) {
			wr.Logger().Warningf("Cannot delete ShardReplication in cell %v for %v/%v: %v", cell, keyspace, shard, err)
		}
	}

	return wr.ts.DeleteShard(ctx, keyspace, shard)
}

// SourceShardDelete will delete a SourceShard inside a shard, by index.
//

View on GitHub (pinned to 01a25a7d17)

Solutions

  1. Read the inner error: if the tablet is a primary with replicas, first reparent or remove the replicas, then re-run DeleteShard -recursive.
  2. Check for topo locks (another workflow like Reshard holding the keyspace lock) and wait/cancel it, then retry.
  3. If the error was transient, simply re-run DeleteShard -recursive; it skips tablets already deleted.

Example fix

// before
vtctldclient DeleteShard -recursive commerce/0
// error: can't delete tablet zone1-100: primary tablet has replicas
// after
vtctldclient PluralTabletsDelete --allow_primary zone1-100 zone1-101   # or remove replicas first
vtctldclient DeleteShard -recursive commerce/0
Defensive patterns

Strategy: validation

Validate before calling

// Before recursive delete, ensure primaries are demoted/removed
tablets, _ := vtctldclientListTablets(keyspace, shard)
for _, t := range tablets {
    if t.Type == "PRIMARY" {
        // demote or delete replicas first
        fmt.Printf("primary %s still present; handle replicas first\n", t.Alias)
    }
}

Try / catch

// Go: re-run DeleteShard on transient tablet-delete failure
err := wr.DeleteShard(ctx, ks, shard, cells, true)
if err != nil && strings.Contains(err.Error(), "can't delete tablet") {
    // check topo locks, resolve, then retry (safe: skips already-deleted tablets)
    err = wr.DeleteShard(ctx, ks, shard, cells, true)
}

Prevention

When it happens

Trigger: Recursive DeleteShard where wr.DeleteTablet(alias) returns an error for a tablet — typically deleting a primary tablet that still has serving replicas, or a transient topo write failure.

Common situations: Trying to tear down a shard whose primary still has followers; locked topo server (another vtctld operation holds the shard/keyspace lock); network blip mid-deletion.

Related errors

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/06cdaa2d3b3dacdb. Report an issue: GitHub.