mastra-ai/mastra · warning

Stale broker election lock removed

Error message

Stale broker election lock removed

What it means

When electing a broker, the process tries to create the election lock file exclusively (open with 'wx'). If it already exists (EEXIST) and is detected as stale (its owner is gone), the library unlinks the lock and throws this sentinel error so #start can retry election from scratch.

Source

Thrown at packages/core/src/events/unix-socket-pubsub.ts:446

      }
    }
  }

  /**
   * Serializes broker election across processes using an exclusive lock file.
   * Only the lock winner unlinks the stale socket and listens; losers wait
   * then connect as clients to the newly elected broker.
   */
  async #electBroker(): Promise<void> {
    const lockPath = this.socketPath + '.elect';
    let lockFd: FileHandle | undefined;
    try {
      lockFd = await open(lockPath, 'wx');
    } catch (e) {
      if ((e as NodeJS.ErrnoException).code === 'EEXIST') {
        if (await this.#isElectionLockStale(lockPath)) {
          await unlink(lockPath).catch(() => {});
          throw new Error('Stale broker election lock removed');
        }
        await new Promise(resolve => setTimeout(resolve, 150));
        try {
          await this.#connectClient();
          this.#throwIfClosed();
          return;
        } catch {
          throw new Error('Broker election in progress by another process');
        }
      }
      throw e;
    }

    try {
      // Re-check: a previous election round may have installed a broker
      // between our initial connectClient() and acquiring this lock.
      try {
        await this.#connectClient();

View on GitHub (pinned to 75dd419e61)

Solutions

1. Simply retry the operation (publish/subscribe) — the library removed the stale lock and the next #ensureStarted/#start attempt can win the election
2. If it persists, manually remove the stale lock file at the socket path
3. Ensure the socket/lock directory is writable and not reused by a dead container without cleanup
4. Align startup so one process elects before others connect

Example fix

// before
await pubsub.publish("t", e); // threw 'Stale broker election lock removed'
// after
try {
  await pubsub.publish("t", e);
} catch {
  await pubsub.publish("t", e); // retry: stale lock was cleared
}

// pre-check lock dir before startup
const lockPath = getElectionLockPath(socketPath);
if (existsSync(lockPath) && !(await isLockFresh(lockPath))) {
  rmSync(lockPath, { force: true });
}

try {
  await pubsub.publish(topic, event);
} catch (e) {
  if (e.message === 'Stale broker election lock removed') {
    await waitFor(250);
    return retryOnce(); // election can now proceed
  }
  throw e;
}

When it happens

Trigger: #electBroker (from #start) finds lockPath exists via open(...,'wx') EEXIST and #isElectionLockStale returns true — e.g. after a previous process crashed while holding the lock. It unlinks the file and throws to restart the election.

Common situations: A previous run crashed or was SIGKILLed without releasing the election lock, container restarts leaving the socket directory on a volume, multiple processes starting after an abrupt termination.

Related errors

• Broker election in progress by another process
• SignalsPubSub is closed
• UnixSocketPubSub does not support grouped subscriptions yet
• UnixSocketPubSub is closed
• Factory kickoff run ended in error.

AI-assisted analysis of mastra-ai/mastra@75dd419e61 (2026-08-30). Data as JSON: /api/errors/048856787b9ffb60. Report an issue: GitHub.