stablyai/orca · error · AggregateError

Benchmark pending anchor recovery failed

Error message

Benchmark pending anchor recovery failed

What it means

Thrown by signalValidatedProcessGroup when resuming a previously-SIGSTOP'd anchor process (groupState.anchorPid) fails with an error other than ESRCH. The underlying signal error is wrapped in an AggregateError so the original cause survives. ESRCH is tolerated because a vanished process is the expected race during teardown; any other failure (EPERM, EINVAL) means the anchor is in an unexpected state and the trial must abort.

Source

Thrown at config/scripts/macos-computer-helper-owner-loss-processes.mjs:271

    const ownershipError = new Error('Benchmark process group no longer belongs to this trial')
    const recoveryErrors = compensateStoppedGroup(pgid, groupState, operations)
    if (recoveryErrors.length > 0) {
      throw new AggregateError(
        [ownershipError, ...recoveryErrors],
        'Benchmark process group authority recovery failed'
      )
    }
    throw ownershipError
  }
  if (groupState.anchorPid) {
    try {
      operations.signalProcess(groupState.anchorPid, 'SIGCONT')
      groupState.anchorPid = null
    } catch (error) {
      if (error?.code === 'ESRCH') {
        groupState.anchorPid = null
      } else {
        throw new AggregateError([error], 'Benchmark pending anchor recovery failed')
      }
    }
  }
  const anchor = members[0]
  try {
    operations.signalProcess(anchor.pid, 'SIGSTOP')
    groupState.anchorPid = anchor.pid
    const stoppedAnchor = operations
      .processIdentities(true)
      .find((identity) => identity.pid === anchor.pid)
    if (!sameIdentity(stoppedAnchor, anchor)) {
      throw new Error('Benchmark process group anchor changed before signaling')
    }
    operations.signalProcess(-pgid, 'SIGSTOP')
    groupState.stopped = true
    groupState.anchorPid = null
    const stoppedMembers = operations
      .processIdentities(true)

View on GitHub (pinned to 1136503c6a)

Solutions

1. Reset groupState to { stopped: false, anchorPid: null } between independent trials so no stale anchor is resumed
2. Before the second call, verify the anchor is still owned by the benchmark user via processIdentity plus a uid check
3. In tests, make the stubbed signalProcess rethrow only ESRCH-coded errors for dead pids and swallow nothing else
4. Catch AggregateError at the trial boundary and run compensateStoppedGroup followed by killProcessMatchingCommand during cleanup

Example fix

// before
signalValidatedProcessGroup(pgid, frag, 'SIGTERM', groupState)
// groupState.anchorPid left set from the prior run -> SIGCONT throws EPERM

// after
const groupState = { stopped: false, anchorPid: null }
signalValidatedProcessGroup(pgid, frag, 'SIGTERM', groupState)

function safeToResumeAnchor(groupState, operations) {
  if (!groupState.anchorPid) return true
  const id = operations.processIdentity(groupState.anchorPid)
  return id && id.command.includes(environmentFragment)
}
// call before signalValidatedProcessGroup:
if (!safeToResumeAnchor(groupState, processGroupSignalOperations)) {
  groupState.anchorPid = null // give up on stale anchor
}

function isAggregateError(e) {
  return e instanceof Error && Array.isArray(e.errors)
}

try {
  signalValidatedProcessGroup(pgid, frag, sig, groupState)
} catch (error) {
  const causes = error instanceof AggregateError ? error.errors : [error]
  for (const c of causes) log.error(c)
  killProcessMatchingCommand([frag])
}

When it happens

Trigger: A prior call left groupState.anchorPid set (anchor SIGSTOP'd), and the follow-up operations.signalProcess(anchorPid, 'SIGCONT') throws a non-ESRCH error. Concretely: the anchor's owning uid changed (EPERM), the pid is invalid (EINVAL), or a test-injected operations.signalProcess stub threw without an ESRCH code.

Common situations: The macOS owner-loss scenario the file is named for — a helper process's owner changes between trials and resuming it raises EPERM. Also seen with a groupState object reused across trials without being reset, or with a signalProcess stub in unit tests that does not emulate ESRCH for dead pids.

Related errors

• Benchmark process group signal recovery failed
• Benchmark helper signal recovery failed
• Expected helper to exit after abrupt authenticated owner los
• Expected baseline helper to remain after abrupt owner loss
• Benchmark process group anchor changed before signaling

AI-assisted analysis of stablyai/orca@1136503c6a (2026-08-12). Data as JSON: /api/errors/280536e4652bd370. Report an issue: GitHub.