{"record":{"id":"048856787b9ffb60","repo":"mastra-ai/mastra","slug":"stale-broker-election-lock-removed","errorCode":null,"errorMessage":"Stale broker election lock removed","messagePattern":"Stale broker election lock removed","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"packages/core/src/events/unix-socket-pubsub.ts","lineNumber":446,"sourceCode":"      }\n    }\n  }\n\n  /**\n   * Serializes broker election across processes using an exclusive lock file.\n   * Only the lock winner unlinks the stale socket and listens; losers wait\n   * then connect as clients to the newly elected broker.\n   */\n  async #electBroker(): Promise<void> {\n    const lockPath = this.socketPath + '.elect';\n    let lockFd: FileHandle | undefined;\n    try {\n      lockFd = await open(lockPath, 'wx');\n    } catch (e) {\n      if ((e as NodeJS.ErrnoException).code === 'EEXIST') {\n        if (await this.#isElectionLockStale(lockPath)) {\n          await unlink(lockPath).catch(() => {});\n          throw new Error('Stale broker election lock removed');\n        }\n        await new Promise(resolve => setTimeout(resolve, 150));\n        try {\n          await this.#connectClient();\n          this.#throwIfClosed();\n          return;\n        } catch {\n          throw new Error('Broker election in progress by another process');\n        }\n      }\n      throw e;\n    }\n\n    try {\n      // Re-check: a previous election round may have installed a broker\n      // between our initial connectClient() and acquiring this lock.\n      try {\n        await this.#connectClient();","sourceCodeStart":428,"sourceCodeEnd":464,"githubUrl":"https://github.com/mastra-ai/mastra/blob/75dd419e613fe9c39f846ffc500716141b74fda6/packages/core/src/events/unix-socket-pubsub.ts#L428-L464","documentation":"When electing a broker, the process tries to create the election lock file exclusively (open with 'wx'). If it already exists (EEXIST) and is detected as stale (its owner is gone), the library unlinks the lock and throws this sentinel error so #start can retry election from scratch.","triggerScenarios":"#electBroker (from #start) finds lockPath exists via open(...,'wx') EEXIST and #isElectionLockStale returns true — e.g. after a previous process crashed while holding the lock. It unlinks the file and throws to restart the election.","commonSituations":"A previous run crashed or was SIGKILLed without releasing the election lock, container restarts leaving the socket directory on a volume, multiple processes starting after an abrupt termination.","solutions":["Simply retry the operation (publish/subscribe) — the library removed the stale lock and the next #ensureStarted/#start attempt can win the election","If it persists, manually remove the stale lock file at the socket path","Ensure the socket/lock directory is writable and not reused by a dead container without cleanup","Align startup so one process elects before others connect"],"exampleFix":"// before\nawait pubsub.publish(\"t\", e); // threw 'Stale broker election lock removed'\n// after\ntry {\n  await pubsub.publish(\"t\", e);\n} catch {\n  await pubsub.publish(\"t\", e); // retry: stale lock was cleared\n}","handlingStrategy":"retry","validationCode":"// pre-check lock dir before startup\nconst lockPath = getElectionLockPath(socketPath);\nif (existsSync(lockPath) && !(await isLockFresh(lockPath))) {\n  rmSync(lockPath, { force: true });\n}","typeGuard":null,"tryCatchPattern":"try {\n  await pubsub.publish(topic, event);\n} catch (e) {\n  if (e.message === 'Stale broker election lock removed') {\n    await waitFor(250);\n    return retryOnce(); // election can now proceed\n  }\n  throw e;\n}","preventionTips":["Retry the first publish/subscribe with short backoff during startup","Ensure previous processes release/cleanup locks on SIGTERM","Clean socket/lock dirs in container entrypoints before start","Avoid SIGKILL termination for processes using UnixSocketPubSub"],"tags":["pubsub","unix-socket","broker-election","stale-lock","retry"],"backgroundTag":"stale-lock-file","analyzedSha":"75dd419e613fe9c39f846ffc500716141b74fda6","analyzedAt":"2026-08-30T00:15:31.844Z","schemaVersion":2},"datasetVersion":"2026-08-30T03:17:51.788Z"}