vitessio/vitess · error

gave up waiting for mysqld to stop

Error message

gave up waiting for mysqld to stop

What it means

waitForMysqldExit polls the socket and pid files while shutting mysqld down; shutdown succeeds when both disappear. If the context is cancelled or its deadline expires while mysqld is still running (files still present), this error reports that mysqld refused to exit in time. It signals a hung or wedged shutdown rather than a failed command invocation.

Source

Thrown at go/vt/mysqlctl/mysqld.go:1241

// message, the match returns false and Shutdown reports the mysqladmin
// error exactly as it did before this check existed. The endtoend mysqlctl
// suite exercises this match against the real mysqladmin binary.
func mysqladminAbortedWaiting(output string) bool {
	return strings.Contains(output, "Aborted waiting on pid file")
}

// waitForMysqldExit polls until both socketFile and pidFile have been removed,
// which signals that the mysqld process has fully exited.
func waitForMysqldExit(ctx context.Context, socketFile, pidFile string) error {
	for {
		_, socketErr := os.Stat(socketFile)
		_, pidErr := os.Stat(pidFile)
		if os.IsNotExist(socketErr) && os.IsNotExist(pidErr) {
			return nil
		}
		select {
		case <-ctx.Done():
			return errors.New("gave up waiting for mysqld to stop")
		case <-time.After(mysqldExitPollInterval):
		}
	}
}

// execCmd searches the PATH for a command and runs it, logging the output.
// If input is not nil, pipe it to the command's stdin. It runs without a
// deadline; use execCmdWithContext to bound the command by a context.
func execCmd(name string, args, env []string, dir string, input io.Reader) (cmd *exec.Cmd, output string, err error) {
	return execCmdWithContext(context.Background(), name, args, env, dir, input)
}

// execCmdWithContext searches the PATH for a command and runs it, logging the
// output. If input is not nil, pipe it to the command's stdin. If ctx is
// cancelled or its deadline passes, the command is killed and the call returns
// promptly rather than blocking on a stalled process.
func execCmdWithContext(ctx context.Context, name string, args, env []string, dir string, input io.Reader) (cmd *exec.Cmd, output string, err error) {
	cmdPath, _ := exec.LookPath(name)

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Increase the shutdown context deadline to accommodate slow mysqld shutdowns
2. Check mysqld's error log to see what is blocking shutdown (long transactions, slow flush)
3. Use a forced/immediate shutdown mode if clean shutdown is not required
4. Ensure no client connections or long-running queries keep mysqld alive during shutdown

Example fix

// before
ctx, cancel := context.WithTimeout(ctx, 3*time.Second)
err := mysqld.Shutdown(ctx, cnf, true) // gave up waiting
// after
ctx, cancel := context.WithTimeout(ctx, 120*time.Second)
err := mysqld.Shutdown(ctx, cnf, true)

// before shutdown, ensure no blockers
if pid, err := os.ReadFile(mycnf.PidFile); err == nil {
    log.Info("shutting down mysqld", slog.String("pid", strings.TrimSpace(string(pid))))
}

ctx, cancel := context.WithTimeout(ctx, 120*time.Second)
defer cancel()
if err := mysqld.Shutdown(ctx, mycnf, false); err != nil {
    if strings.Contains(err.Error(), "gave up waiting for mysqld to stop") {
        log.Error("mysqld shutdown timeout; check error log and consider forced shutdown", slog.Any("error", err))
    }
    return err
}

When it happens

Trigger: Calling Shutdown (via executeShutdown) or StartAfterExit with a context that expires before the mysqld process writes no more and removes its socket/pid files; mysqld stuck in a long rollback or waiting on a slow shutdown.

Common situations: Large InnoDB buffers making a clean shutdown slow; mysqld ignoring SIGTERM and requiring a longer grace period; shutdown context deadlines set too aggressively in tests or CI.

Related errors

• deadline exceeded waiting for mysqld socket file to appear:
• ReadFile cannot be called on read-write backup
• AddFile cannot be called on read-only backup
• EndBackup cannot be called on read-only backup
• AbortBackup cannot be called on read-only backup

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/ada484f6d7749420. Report an issue: GitHub.