koala73/worldmonitor · error · Error

[clearPartialFailure] refused: pending progress markers pres

Error message

[clearPartialFailure] refused: pending progress markers present (waveLabel=${row.pendingWaveLabel ?? "-"}, segmentId=${row.pendingSegmentId ?? "-"}, broadcastId=${row.pendingBroadcastId ?? "-"}). The export DID run; clearing here would mask stamped contacts. Use recoverFromPartialFailure instead.

What it means

clearPartialFailure is a fail-closed guard: if any pending-progress marker (pendingWaveLabel, pendingSegmentId, pendingBroadcastId) is present, the export side of the wave actually executed and contacts may already be stamped. Clearing would mask a partially-sent wave. The handler throws and directs the operator to recoverFromPartialFailure instead.

Source

Thrown at convex/broadcast/rampRunner.ts:313

  handler: async (ctx, { reason }) => {
    const row = await loadConfig(ctx);
    if (!row) throw new Error("[clearPartialFailure] no ramp configured");
    if (row.lastRunStatus !== "partial-failure") {
      return {
        ok: true as const,
        noop: true as const,
        currentStatus: row.lastRunStatus,
      };
    }
    // Fail-closed: if any pending-progress marker exists, the export DID make
    // progress past `assignAndExportWave` — clearing here would mask a stamped
    // / sent wave. Force the operator to use recoverFromPartialFailure.
    if (
      row.pendingWaveLabel ||
      row.pendingSegmentId ||
      row.pendingBroadcastId
    ) {
      throw new Error(
        `[clearPartialFailure] refused: pending progress markers present (waveLabel=${row.pendingWaveLabel ?? "-"}, segmentId=${row.pendingSegmentId ?? "-"}, broadcastId=${row.pendingBroadcastId ?? "-"}). The export DID run; clearing here would mask stamped contacts. Use recoverFromPartialFailure instead.`,
      );
    }
    await ctx.db.patch(row._id, {
      lastRunStatus: `partial-failure-cleared: ${reason.slice(0, 200)}`,
      lastRunError: undefined,
      pendingRunId: undefined,
      pendingRunStartedAt: undefined,
    });
    return { ok: true as const };
  },
});

/**
 * Structured recovery for `lastRunStatus === "partial-failure"` that ALSO
 * occurred AFTER `assignAndExportWave` succeeded (or after a forced lease
 * release on a wedged run). Two recovery modes:
 *

View on GitHub (pinned to ffec79ac33)

Solutions

  1. Do not clear. Inspect the persisted pending* markers (getRampStatus / direct table read) to see how far the wave got.
  2. Use recoverFromPartialFailure({recovery:'manual-finished', ...}) if the broadcast actually sent, supplying broadcastId/segmentId/assigned/sentAt.
  3. Use recoverFromPartialFailure({recovery:'discard-and-rotate'}) only if you have verified the broadcast did NOT go out and you accept rotating the wave offset.

Example fix

// before
await ctx.runMutation(internal.broadcast.rampRunner.clearPartialFailure, { reason, confirmNoExport: true });

// after — pending markers present means the export ran; recover, do not clear
await ctx.runMutation(internal.broadcast.rampRunner.recoverFromPartialFailure, {
  recovery: 'manual-finished',
  reason,
  broadcastId: row.pendingBroadcastId,
  segmentId: row.pendingSegmentId,
  assigned: row.pendingAssigned,
  sentAt: confirmedSendTimestamp,
});
Defensive patterns

Strategy: validation

Validate before calling

// Before clearPartialFailure, detect pending-progress markers and route to recovery
const row = await ctx.db.query('broadcastRampConfig').withIndex('by_key', (q) => q.eq('key', 'current')).first();
if (row && (row.pendingWaveLabel || row.pendingSegmentId || row.pendingBroadcastId)) {
  throw new Error('Pending progress markers present — use recoverFromPartialFailure, not clearPartialFailure.');
}

Type guard

function hasPendingProgress(row: { pendingWaveLabel?: string; pendingSegmentId?: string; pendingBroadcastId?: string }): boolean {
  return Boolean(row.pendingWaveLabel || row.pendingSegmentId || row.pendingBroadcastId);
}

Prevention

When it happens

Trigger: Calling clearPartialFailure when the failed run had already persisted pendingWaveLabel/pendingSegmentId/pendingBroadcastId via _recordPendingExport or _recordPendingBroadcast; i.e. the runner died after the export/broadcast side effect started but before _recordWaveSent committed.

Common situations: A wave action timed out or the Convex action was interrupted after assignAndExportWave or createProLaunchBroadcast ran; the operator tries the 'easy' clear instead of the proper recovery path.

Related errors

AI-assisted analysis of koala73/worldmonitor@ffec79ac33 (2026-08-12). Data as JSON: /api/errors/4b7f673ba2494613. Report an issue: GitHub.