vitessio/vitess · error

Shutdown didn't kill process %v

Error message

Shutdown didn't kill process %v

What it means

Zkd.Shutdown sends SIGTERM and waits for the pid to die; if after the timeout window a SIGKILL check still finds the process alive (Kill does not return ESRCH), this error is returned. It means zookeeper refused to die within the allotted time.

Source

Thrown at go/vt/zkctl/zkctl.go:153

	if err != nil {
		return err
	}
	pid, err := strconv.Atoi(string(bytes.TrimSpace(pidData)))
	if err != nil {
		return err
	}
	err = syscallutil.Kill(pid, syscall.SIGKILL)
	if err != nil && err != syscall.ESRCH {
		return err
	}
	timeout := time.Now().Add(shutdownWaitTime)
	for time.Now().Before(timeout) {
		if syscallutil.Kill(pid, syscall.SIGKILL) == syscall.ESRCH {
			return nil
		}
		time.Sleep(time.Second)
	}
	return fmt.Errorf("Shutdown didn't kill process %v", pid)
}

func (zkd *Zkd) makeCfg() (string, error) {
	root, err := env.VtRoot()
	if err != nil {
		return "", err
	}
	cnfTemplatePaths := []string{path.Join(root, "config/zkcfg/zoo.cfg")}
	return MakeZooCfg(cnfTemplatePaths, zkd.config, "# generated by vt")
}

// Init generates a new config and then starts ZooKeeper.
func (zkd *Zkd) Init() error {
	if zkd.Inited() {
		return errors.New("zk already inited")
	}

	log.Info("zkd.Init")

View on GitHub (pinned to 01a25a7d17)

Solutions

1. Kill the pid manually: `kill -9 <pid>` and remove the stale pid file
2. Check process state with `ps -o stat -p <pid>` (D/S/T states explain the hang)
3. If disk I/O is stuck, fix the underlying storage or restart the host

if syscallutil.Kill(pid, 0) == nil {
    log.Infof("zk pid %d still running before shutdown attempt", pid)
}

if err := zkd.Shutdown(); err != nil {
    if strings.Contains(err.Error(), "didn't kill process") {
        syscall.Kill(pid, syscall.SIGKILL) // escalate
    }
}

When it happens

Trigger: Calling zkd.Shutdown() (via zkctl Teardown) when the zk process ignores/kills slowly — kill signal delivered but process still present past the timeout deadline.

Common situations: Zookeeper stuck in uninterruptible I/O on a slow/hung disk; process in a stopped (SIGSTOP) state; very slow container under resource pressure; wrong pid recorded in the pid file.

Related errors

• invalid choice for enum
• deadline exceeded waiting for mysqld socket file to appear:
• gave up waiting for mysqld to stop
• gRPC connection wait time exceeded
• can't lock shard: SIGTERM received

AI-assisted analysis of vitessio/vitess@01a25a7d17 (2026-09-01). Data as JSON: /api/errors/317aec53b76d3e49. Report an issue: GitHub.