{"record":{"id":"55b4aa32b84323ed","repo":"paperclipai/paperclip","slug":"refusing-embedded-postgresql-recovery-because-the","errorCode":null,"errorMessage":"Refusing embedded PostgreSQL recovery because the data directory reports a live process (pid=${runningPostgresPid})","messagePattern":"Refusing embedded PostgreSQL recovery because the data directory reports a live process \\(pid=(.+?)\\)","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"critical","filePath":"server/src/index.ts","lineNumber":572,"sourceCode":"            });\n          }\n        } else {\n          logger.info(`Embedded PostgreSQL cluster already exists (${clusterVersionFile}); skipping init`);\n        }\n\n        if (existsSync(postmasterPidFile)) {\n          logger.warn(\"Removing stale embedded PostgreSQL lock file\");\n          rmSync(postmasterPidFile, { force: true });\n        }\n        try {\n          await embeddedPostgres.start();\n        } catch (err) {\n          logEmbeddedPostgresFailure(\"start\", err);\n          throw formatEmbeddedPostgresError(err, {\n            fallbackMessage: `Failed to start embedded PostgreSQL on port ${port}`,\n            recentLogs: logBuffer.getRecentLogs(),\n          });\n        }\n        embeddedPostgresStartedByThisProcess = true;\n        embeddedPostgresSupervisor = createEmbeddedPostgresSupervisor({\n          initialInstance: embeddedPostgres,\n          createInstance: createEmbeddedPostgres,\n          beforeRestart: () => {\n            const runningPostgresPid = getRunningPid();\n            if (runningPostgresPid) {\n              throw new Error(`Refusing embedded PostgreSQL recovery because the data directory reports a live process (pid=${runningPostgresPid})`);\n            }\n            if (existsSync(postmasterPidFile)) rmSync(postmasterPidFile, { force: true });\n          },\n          onUnexpectedExit: (code, signal) => logger.error(\n            { code, signal, recentLogs: logBuffer.getRecentLogs() },\n            \"Embedded PostgreSQL exited unexpectedly; attempting recovery\",\n          ),\n          onRestartAttemptFailed: (err, attempt) => logger.error(\n            { err, attempt, recentLogs: logBuffer.getRecentLogs() },\n            \"Embedded PostgreSQL recovery attempt failed\",","sourceCodeStart":554,"sourceCodeEnd":590,"githubUrl":"https://github.com/paperclipai/paperclip/blob/01ad8584922b5d85292b1723cae71fa0d9b07a19/server/src/index.ts#L554-L590","documentation":"When embedded PostgreSQL exits unexpectedly, the supervisor tries to recover by restarting it, but first re-checks the postmaster.pid in the data directory. If the pid recorded there is still a live process (process.kill(pid, 0) succeeds), restarting could corrupt a data directory that another running postmaster owns, so recovery is refused. This is a safety stop, not a transient failure.","triggerScenarios":"beforeRestart hook in createEmbeddedPostgresSupervisor (server startup with no DATABASE_URL, embedded PG mode): embedded PG crashed or 'exited' from this process's view while a postmaster with the pid in postmaster.pid is still alive — e.g. a forked/orphaned postmaster, another Paperclip server instance sharing the data dir, or the pid got reused by an unrelated live process.","commonSituations":"Two dev servers started against the same PGlite/embedded data directory; a previous server killed with SIGKILL leaving the postmaster alive detached; container/PID-namespace weirdness where the recorded host pid maps to a live process; pid reuse on long-running machines after an unclean shutdown.","solutions":["Identify the process: `ps -p <pid> -o pid,ppid,command` — confirm whether it is really a postgres postmaster.","If it is a leftover Paperclip/postgres you own, shut it down gracefully (`pg_ctl stop -D <dataDir>` or stop the other Paperclip process), then restart the server so recovery can proceed.","If the pid belongs to an unrelated process (pid reuse), verify no postgres owns the data dir, then remove the stale postmaster.pid manually and restart.","Prevent recurrence: run only one embedded-mode server per data directory; prefer a managed Postgres via DATABASE_URL for parallel dev servers."],"exampleFix":"# before: recovery refused, pid 4242 alive\nps -p 4242 -o pid,command   # leftover postmaster from a killed server\npg_ctl stop -D ~/.paperclip/embedded-pg/data || kill 4242\n\n# after: pid gone, restart the Paperclip server; supervisor recovery proceeds\nrm -f ~/.paperclip/embedded-pg/data/postmaster.pid   # only if ps shows no postgres\npaperclip dev","handlingStrategy":"fallback","validationCode":"import { existsSync, readFileSync } from \"node:fs\";\nimport path from \"node:path\";\n\nfunction pidAlive(pid: number): boolean {\n  try { process.kill(pid, 0); return true; } catch { return false; }\n}\n\nfunction embeddedPgSafeToStart(dataDir: string): boolean {\n  const pidFile = path.join(dataDir, \"postmaster.pid\");\n  if (!existsSync(pidFile)) return true;\n  const pid = Number(readFileSync(pidFile, \"utf8\").split(\"\\n\")[0]);\n  return !Number.isInteger(pid) || pid <= 0 || !pidAlive(pid);\n}","typeGuard":"const isPostmasterPidStale = (dataDir: string): boolean => {\n  const pidFile = path.join(dataDir, \"postmaster.pid\");\n  if (!existsSync(pidFile)) return false;\n  const pid = Number(readFileSync(pidFile, \"utf8\").split(\"\\n\")[0]);\n  try { process.kill(pid, 0); return false; } catch { return true; }\n};","tryCatchPattern":null,"preventionTips":["Run only one embedded-mode Paperclip server per data directory.","Shut down dev servers gracefully (SIGTERM) so postmaster cleans its pid file.","For parallel dev servers, set DATABASE_URL to separate managed Postgres instances.","On boot failures, check `ps aux | grep postgres` and the embedded PG log before deleting postmaster.pid."],"tags":["postgres","embedded-database","process-management","recovery","dev-server"],"backgroundTag":"postgres-postmaster-pid-conflict","analyzedSha":"01ad8584922b5d85292b1723cae71fa0d9b07a19","analyzedAt":"2026-08-21T17:58:32.592Z","contentChangedAt":"2026-08-21T17:58:32.592Z","schemaVersion":2},"datasetVersion":"2026-09-14T00:17:10.932Z"}