mastra-ai/mastra · error

Factory kickoff is waiting on a run that has not ended.

Error message

Factory kickoff is waiting on a run that has not ended.

What it means

The dispatcher attempted a kickoff against a run that was still in flight. It waited for the agent to end via waitForAgentEndOrTimeout using #skillCompletionObservationTimeoutMs; if the run did not produce a terminal event within that window, the library throws so the pending start fails and the record is retried with an incremented attempts count and a new retry-suffixed kickoff key.

Source

Thrown at mastracode/factory/src/rules/dispatcher.ts:982

                    dedupeKey,
                  },
                  { ifActive: { behavior: 'deliver' }, ifIdle: { behavior: 'wake' }, requestContext },
                ),
              true,
            );
          try {
            let settled = await sendKickoff(`factory-kickoff:${record.kickoffKey}`);
            if (settled?.action === 'deliver') {
              // `deliver` only proves the signal was queued onto a run already
              // in flight. If that run ends without draining its queue the
              // kickoff is dropped silently. There is no per-notification
              // "processed" signal, so wait for the in-flight run to end and
              // redeliver into the idle session unconditionally — the
              // generation-scoped dedupeKey defeats inbox dedupe and the
              // kickoff key keeps a duplicate run bounded, while a dropped
              // kickoff strands the card forever.
              if (!(await waitForAgentEndOrTimeout(agentEnd, this.#skillCompletionObservationTimeoutMs))) {
                throw new Error('Factory kickoff is waiting on a run that has not ended.');
              }
              armAgentEnd();
              settled = await sendKickoff(`factory-kickoff:${record.kickoffKey}:retry:${record.attempts}`);
              if (settled?.action !== 'wake') {
                throw new Error('Factory kickoff was queued onto an ending run and never reached the agent.');
              }
            }
            const observed = await waitForAgentEndOrTimeout(agentEnd, this.#skillCompletionObservationTimeoutMs);
            if (!observed) {
              throw new Error('Factory kickoff run terminal event was not observed before timeout.');
            } else if (endReason === 'error') {
              throw new Error('Factory kickoff run ended in error.');
            } else if (endReason === 'aborted') {
              // Retryable for the same reason as skill decisions: the dominant
              // cause is the process going away underneath the run, not a
              // deliberate stop, and a spurious retry is bounded by
              // MAX_ATTEMPTS while a dead card costs a human a manual nudge.
              throw new Error('Factory kickoff run was aborted before it finished.');

View on GitHub (pinned to 75dd419e61)

Solutions

1. Increase skillCompletionObservationTimeoutMs (or the equivalent config) to exceed realistic run durations for your models.
2. Investigate why the run hangs — check model provider health, stream liveness, and trace the agent run for stalls.
3. Let the retry machinery re-deliver the kickoff (retry key factory-kickoff:<key>:retry:<attempts>) after the run ends.
4. Reduce concurrent load on the session so runs complete within the observation window.

Example fix

// before: timeout too small for long generations
new FactoryDecisionDispatcher({ skillCompletionObservationTimeoutMs: 5_000, ... });
// after: allow realistic run completion time
new FactoryDecisionDispatcher({ skillCompletionObservationTimeoutMs: 120_000, ... });

const runActive = await isRunActive(sessionId); // check run state before kickoff
if (runActive && expectedRunDurationMs > observationTimeoutMs) {
  throw new Error('run likely to outlive kickoff observation window');
}

try {
  await dispatcher.dispatch(record);
} catch (e) {
  if (String(e?.message).includes('waiting on a run that has not ended')) {
    await scheduleRetry(record); // dispatcher already backs off via attempts
  } else throw e;
}

When it happens

Trigger: An agent run on the target session outlived skillCompletionObservationTimeoutMs (long-running generation, hung model call, streaming stalls) so the observed agentEnd promise never resolved before the timeout.

Common situations: Model provider latency or stalled streams exceeding the observation timeout; timeout configured too aggressively for slow models; a run stuck because the process is under heavy load; debugging/breakpoints holding the run open during development.

Related errors

• Factory kickoff run terminal event was not observed before t
• Factory kickoff was queued onto an ending run and never reac
• Factory kickoff run ended in error.
• MastraAuthBetterAuth is not initialized — init() must run fi
• State token has expired

AI-assisted analysis of mastra-ai/mastra@75dd419e61 (2026-08-30). Data as JSON: /api/errors/d8d5bc5ca68f6d48. Report an issue: GitHub.