failed company transfer runs stranded in applying by a resta

Error message

failed company transfer runs stranded in applying by a restart

What it means

Company-transfer apply jobs are in-memory in the single server process. At startup, recoverStrandedApplyingRuns fails any run still marked 'applying' — those were interrupted by the previous shutdown and would otherwise 409 on every retry forever. Their spooled transfer parts remain reusable, and this warning lists the recovered run ids.

Source

Thrown at server/src/app.ts:849

      .catch((err) => {
        logger.error({ err }, "abandoned company import transfer spool sweep failed");
      });
  };
  let importTransferSweepTimer: ReturnType<typeof setInterval> | null = setInterval(
    sweepImportTransferSpools,
    IMPORT_TRANSFER_SPOOL_SWEEP_INTERVAL_MS,
  );
  importTransferSweepTimer.unref?.();
  // Startup only (never on the hourly interval — that would kill live
  // applies): apply jobs are in-memory in this single process, so any run
  // still "applying" now was interrupted by the previous shutdown and would
  // otherwise 409 every retry forever. Fail those stranded runs — their
  // spooled parts stay reusable — then run the normal sweep once.
  void companyTransferRunService
    .recoverStrandedApplyingRuns(db)
    .then((recovered) => {
      if (recovered.length > 0) {
        logger.warn(
          { count: recovered.length, runIds: recovered },
          "failed company transfer runs stranded in applying by a restart",
        );
      }
    })
    .catch((err) => {
      logger.error({ err }, "stranded company transfer apply recovery failed");
    })
    .finally(() => {
      sweepImportTransferSpools();
    });
  void toolDispatcher.initialize().catch((err) => {
    logger.error({ err }, "Failed to initialize plugin tool dispatcher");
  });
  const devWatcher = createPluginDevWatcher(
    lifecycle,
    async (pluginId) => (await pluginRegistry.getById(pluginId))?.packagePath ?? null,
  );

View on GitHub (pinned to a7e689b3c3)

Solutions

Retry the company transfers whose runIds appear in the warning — spooled parts make the retry cheap.
Before planned restarts, let in-flight transfers finish (maintenance window / drain) to avoid stranding.
If the restart was unexpected, check the previous process's logs for the crash cause.
Do not hand-edit the runs' DB state — recovery already failed them cleanly.

Defensive patterns

Strategy: retry

Prevention

Quiesce company transfers (drain/finish) before planned restarts and deploys.
Keep the transfer spool directory on durable storage so retried runs reuse parts.
Alert on this startup warn and retry the listed runIds — recovery already failed them cleanly.
Repeated strandings indicate a crash loop; investigate the shutdown cause, not the warn.

When it happens

Trigger: The server process restarted (deploy, crash, OOM kill, container restart) while a company import transfer was in its 'applying' phase; on next boot those runs are failed and listed in the warn.

Common situations: Rolling deploys landing during active imports; crashes mid-apply; aggressive container restart policies; host reboots during bulk company imports.

Related errors

AI-assisted analysis of paperclipai/paperclip@a7e689b3c3 (2026-08-18). Data as JSON: /api/errors/c97d5768b9523854. Report an issue: GitHub.