koala73/worldmonitor · error · Error

[_recordWaveSent] tier moved underneath us: expected ${args.

Error message

[_recordWaveSent] tier moved underneath us: expected ${args.expectedCurrentTier}, found ${row.currentTier}. Refusing to overwrite.

What it means

_recordWaveSent asserts broadcastRampConfig.currentTier still equals the expectedCurrentTier the run was started with. A mismatch means another run already advanced (or rewound) the tier underneath this one — committing would overwrite the wrong tier's state or double-advance. The handler refuses and surfaces to Sentry.

Source

Thrown at convex/broadcast/rampRunner.ts:784

 * record a successful wave-send. Validates that the lease still belongs to
 * this runId AND clears all pending-progress markers.
 */
export const _recordWaveSent = internalMutation({
  args: {
    runId: v.string(),
    expectedCurrentTier: v.number(),
    newTier: v.number(),
    waveLabel: v.string(),
    broadcastId: v.string(),
    segmentId: v.string(),
    assigned: v.number(),
    sentAt: v.number(),
  },
  handler: async (ctx, args) => {
    const row = await loadConfig(ctx);
    if (!row) throw new Error("[_recordWaveSent] no ramp configured");
    if (row.currentTier !== args.expectedCurrentTier) {
      throw new Error(
        `[_recordWaveSent] tier moved underneath us: expected ${args.expectedCurrentTier}, found ${row.currentTier}. Refusing to overwrite.`,
      );
    }
    if (row.pendingRunId !== args.runId) {
      // The lease changed under us — operator force-released it, or
      // recoverFromPartialFailure cleared it. We must NOT advance the tier;
      // bubble to Convex auto-Sentry so ops can investigate.
      throw new Error(
        `[_recordWaveSent] lease lost: expected runId=${args.runId}, found ${row.pendingRunId ?? "<cleared>"}. Refusing to advance tier — investigate what cleared the lease.`,
      );
    }
    await ctx.db.patch(row._id, {
      currentTier: args.newTier,
      lastWaveLabel: args.waveLabel,
      lastWaveBroadcastId: args.broadcastId,
      lastWaveSegmentId: args.segmentId,
      lastWaveAssigned: args.assigned,
      lastWaveSentAt: args.sentAt,

View on GitHub (pinned to ffec79ac33)

Solutions

  1. Investigate current currentTier vs the run's expected tier — do not blindly re-commit.
  2. If another run legitimately advanced, discard this run's result (it's stale).
  3. If the broadcast actually sent but the tier moved, use recoverFromPartialFailure(manual-finished) to reconcile rather than re-running _recordWaveSent.
Defensive patterns

Strategy: try-catch

Validate before calling

// Runner-side: re-verify tier unchanged before terminal commit
const row = await ctx.db.query('broadcastRampConfig').withIndex('by_key', (q) => q.eq('key', 'current')).first();
if (!row || row.currentTier !== expectedCurrentTier) {
  return { aborted: 'tier-moved', current: row?.currentTier };
}

Type guard

function tierMatches(row: { currentTier: number } | null, expected: number): row is { currentTier: number } {
  return row !== null && row.currentTier === expected;
}

Try / catch

try {
  await ctx.runMutation(internal.broadcast.rampRunner._recordWaveSent, { runId, expectedCurrentTier, ... });
} catch (e) {
  if ((e as Error).message.includes('tier moved')) {
    // another run advanced the tier — do not re-commit; reconcile via recoverFromPartialFailure
    return { aborted: 'tier-moved', detail: (e as Error).message };
  }
  throw e;
}

Prevention

When it happens

Trigger: Two runs both passed the lease claim on the same tier (race) and one already committed; an operator used recoverFromPartialFailure which advanced currentTier; abortRamp+initRamp reset the tier while a run was in flight.

Common situations: Concurrent cron + manual run; a recovered partial-failure advanced the tier while the original run was still alive; the ramp was re-seeded mid-flight.

Related errors

AI-assisted analysis of koala73/worldmonitor@ffec79ac33 (2026-08-12). Data as JSON: /api/errors/d68ae3a14ed02968. Report an issue: GitHub.