vitessio/vitess · error

Tablet: %v, is already drained

Error message

Tablet: %v, is already drained

What it means

changeTypeLocked refuses to mark a tablet as DRAINED if the tablet is already in DRAINED state, to prevent multiple concurrent callers from both claiming the drained state. Callers include ChangeType, Backup, InitPrimary, InitReplica and ReplicaWasPromoted.

Source

Thrown at go/vt/vttablet/tabletmanager/rpc_actions.go:141

func (tm *TabletManager) ChangeType(ctx context.Context, tabletType topodatapb.TabletType, semiSync bool) error {
	if err := tm.lock(ctx); err != nil {
		return err
	}
	defer tm.unlock()

	semiSyncAction, err := tm.convertBoolToSemiSyncAction(ctx, semiSync)
	if err != nil {
		return err
	}

	return tm.changeTypeLocked(ctx, tabletType, DBActionNone, semiSyncAction)
}

// changeTypeLocked changes the tablet type under a lock
func (tm *TabletManager) changeTypeLocked(ctx context.Context, tabletType topodatapb.TabletType, action DBAction, semiSync SemiSyncAction) error {
	// We don't want to allow multiple callers to claim a tablet as drained.
	if tabletType == topodatapb.TabletType_DRAINED && tm.Tablet().Type == topodatapb.TabletType_DRAINED {
		return fmt.Errorf("Tablet: %v, is already drained", tm.tabletAlias)
	}

	if err := tm.tmState.ChangeTabletType(ctx, tabletType, action); err != nil {
		return err
	}

	// Let's see if we need to fix semi-sync acking.
	if err := tm.fixSemiSyncAndReplication(ctx, tm.Tablet().Type, semiSync); err != nil {
		return vterrors.Wrap(err, "fixSemiSyncAndReplication failed, may not ack correctly")
	}
	return nil
}

// Sleep sleeps for the duration
func (tm *TabletManager) Sleep(ctx context.Context, duration time.Duration) {
	if err := tm.lock(ctx); err != nil {
		// client gave up
		return

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Wait for the currently running draining operation to finish and release the tablet
2. Restart vttablet to clear a stale DRAINED type left by a crashed operation
3. Check the tablet's current type (vtctldclient GetTablet) before issuing another drain-requiring operation
4. Serialize maintenance operations on a tablet instead of running them concurrently

// Check the tablet is not already drained before requesting a drain-requiring operation
tab := getTablet(t, alias)
if tab.Type == "DRAINED" {
    return fmt.Errorf("tablet %s already drained; wait or restart vttablet", alias)
}

err := withRetry(ctx, func() error { return vtctldclientBackup(alias) })
if err != nil && strings.Contains(err.Error(), "is already drained") {
    // wait for the current operation, then retry
}

When it happens

Trigger: Two concurrent operations that both need to drain the tablet (e.g. two backups, or a backup racing a type change), or a stale tablet record still showing DRAINED from a previously crashed operation.

Common situations: Concurrent vtctldclient commands against the same tablet, a backup running while another maintenance operation starts, leftover DRAINED state after a vttablet crash mid-operation.

Related errors

• a backup is already running on tablet: %v
• invalid choice for enum
• value must be either a float64 (interpreted as seconds) or a
• flagutil: NewOptionalFlag requires a non-nil parse function
• flagutil: OptionalFlagValue has no parse function; use a con

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/c9d8285ef03bd5e3. Report an issue: GitHub.