{"record":{"id":"b7fe4e70ba8c7669","repo":"vitessio/vitess","slug":"process-v-is-still-running","errorCode":null,"errorMessage":"process %v is still running","messagePattern":"process (.+?) is still running","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"go/vt/mysqlctl/mysqld.go","lineNumber":652,"sourceCode":"\t\treturn os.Remove(lockPath)\n\t}\n\tproc, err := os.FindProcess(p)\n\tif err != nil {\n\t\tlog.Error(fmt.Sprintf(\"%v: error finding process: %v\", ts, err))\n\t\treturn err\n\t}\n\terr = proc.Signal(syscall.Signal(0))\n\tif err == nil {\n\t\t// If the process still exists, it's not safe to\n\t\t// remove the lock file, so we have to keep it around.\n\t\tcmdline, err := os.ReadFile(fmt.Sprintf(\"/proc/%d/cmdline\", p))\n\t\tif err == nil {\n\t\t\tname := string(bytes.ReplaceAll(cmdline, []byte{0}, []byte(\" \")))\n\t\t\tlog.Error(fmt.Sprintf(\"%v: not removing socket lock file: %v with pid %v for %q\", ts, lockPath, p, name))\n\t\t} else {\n\t\t\tlog.Error(fmt.Sprintf(\"%v: not removing socket lock file: %v with pid %v (failed to read process name: %v)\", ts, lockPath, p, err))\n\t\t}\n\t\treturn fmt.Errorf(\"process %v is still running\", p)\n\t}\n\tif !errors.Is(err, os.ErrProcessDone) {\n\t\t// Any errors except for the process being done\n\t\t// is unexpected here.\n\t\tlog.Error(fmt.Sprintf(\"%v: error checking process %v: %v\", ts, p, err))\n\t\treturn err\n\t}\n\n\t// All good, process is gone and we can safely clean up the lock file.\n\tlog.Info(fmt.Sprintf(\"%v: removing stale socket lock file: %v\", ts, lockPath))\n\treturn os.Remove(lockPath)\n}\n\n// Wait returns nil when mysqld is up and accepting connections. It\n// will use the dba credentials to try to connect. Use wait() with\n// different credentials if needed.\nfunc (mysqld *Mysqld) Wait(ctx context.Context, cnf *Mycnf) error {\n\tparams, err := mysqld.dbcfgs.DbaConnector().MysqlParams()","sourceCodeStart":634,"sourceCodeEnd":670,"githubUrl":"https://github.com/vitessio/vitess/blob/01a25a7d176f94613b8d59d799f438380a8760e4/go/vt/mysqlctl/mysqld.go#L634-L670","documentation":"cleanupLockfile detected that the PID recorded in the mysqld socket lock file is still alive, so it refuses to delete the lock file and returns 'process %v is still running'. This protects a running mysqld's lock from being stolen by a new instance.","triggerScenarios":"Calling Mysqld.Start/Init (via startNoWait -> cleanupLockfile) when a lock file exists on the target socket path and the PID inside it belongs to a live process (Signal(0)/os.FindProcess check succeeds and err is not os.ErrProcessDone).","commonSituations":"A previous mysqld is genuinely still running on the same socket; orphaned mysqld after an aborted vtaction; PID reuse where a new unrelated process now owns the recorded PID; stale lock from an unclean shutdown whose PID happens to be recycled.","solutions":["Check what process holds the PID: `ps -p <pid>` — if it's a live mysqld, stop it or use a different socket/port","If the PID is reused by an unrelated process, remove the stale lock file manually after confirming mysqld is not running","Clean up orphaned mysqld processes from failed previous starts","Ensure prior vitess workflows terminate mysqld cleanly so locks are removed"],"exampleFix":"// before: start fails, lock file claims pid 1234 alive\nps -p 1234  # old mysqld still running\n// after: stop it, then retry\nmysqladmin -S /path/mysql.sock shutdown\n# or if pid is reused and mysqld is gone:\nrm /path/mysql.sock.lock\n","handlingStrategy":"validation","validationCode":"lockPid := readPidFromLockFile(socketPath + \".lock\")\nif lockPid > 0 {\n    if p, err := os.FindProcess(lockPid); err == nil && p.Signal(syscall.Signal(0)) == nil {\n        return fmt.Errorf(\"pid %d still alive; not safe to start mysqld\", lockPid)\n    }\n}","typeGuard":null,"tryCatchPattern":"if err := mysqld.Start(ctx, cnf); err != nil && strings.Contains(err.Error(), \"is still running\") {\n    // identify the pid from the error and inspect it before removing the lock\n}","preventionTips":["Shut down mysqld cleanly so lock files are removed","Watch for orphaned mysqld after failed workflows","Be aware of PID reuse on long-lived hosts","Use unique sockets/ports per tablet"],"tags":["mysqlctl","startup","lockfile","pid-conflict"],"backgroundTag":"stale-pid-lockfile","analyzedSha":"01a25a7d176f94613b8d59d799f438380a8760e4","analyzedAt":"2026-09-01T17:28:30.605Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}