multica-ai/multica · error

rollup slice %s..%s: %w

Error message

rollup slice %s..%s: %w

What it means

Calling the database function rollup_task_usage_hourly_window(cursor, next) for one monthly slice failed. Each slice runs in its own transaction inside the function, so a failure rolls back only that slice and re-running is safe. Causes include the function not existing (schema migration missing), lock contention/timeout on task_usage_hourly rows, statement failure inside the function, or ctx cancelled between slices.

Source

Thrown at server/cmd/backfill_task_usage_hourly/main.go:168

	slog.Info("backfill range", "from", from.Format(time.RFC3339), "to", end.Format(time.RFC3339), "dry_run", *dryRun, "sleep_between_slices", sleep.String())

	cursor := from
	var totalRows int64
	for cursor.Before(end) {
		next := cursor.AddDate(0, 1, 0)
		if *dryRun {
			slog.Info("would roll up slice", "from", cursor.Format(time.RFC3339), "to", next.Format(time.RFC3339))
			cursor = next
			continue
		}
		var rows int64
		err := pool.QueryRow(
			ctx,
			`SELECT rollup_task_usage_hourly_window($1::timestamptz, $2::timestamptz)`,
			cursor, next,
		).Scan(&rows)
		if err != nil {
			return fmt.Errorf("rollup slice %s..%s: %w", cursor.Format(time.RFC3339), next.Format(time.RFC3339), err)
		}
		totalRows += rows
		slog.Info("rolled up slice", "from", cursor.Format(time.RFC3339), "to", next.Format(time.RFC3339), "rows_touched", rows)
		cursor = next
		if *sleep > 0 && cursor.Before(end) {
			select {
			case <-time.After(*sleep):
			case <-ctx.Done():
				return ctx.Err()
			}
		}
	}

	if *dryRun {
		slog.Info("dry-run complete; watermark left untouched")
		return nil
	}
	// Stamp on a fresh context so a SIGINT arriving after the slices

View on GitHub (pinned to 2c0912b6ec)

Solutions

1. Verify the function exists: \df rollup_task_usage_hourly_window — if not, run migrations first
2. If the wrapped error is context.Canceled, that is a clean SIGINT stop: just re-run; it resumes from the watermark/already-rolled-up slices
3. Raise statement_timeout / lock_timeout for the session, or run with -sleep to throttle slice pressure
4. Check pg_stat_activity for blockers on task_usage_hourly and resolve them

Example fix

# before
backfill_task_usage_hourly -db $DSN   # fails mid-slice on timeout

# after
psql $DSN -c "SET statement_timeout='0'"  # or run with throttling:
backfill_task_usage_hourly -db $DSN -sleep 500ms

var fnExists bool
pool.QueryRow(ctx, `SELECT to_regprocedure('rollup_task_usage_hourly_window(timestamptz,timestamptz)') IS NOT NULL`).Scan(&fnExists)

if err != nil {
    return fmt.Errorf("rollup slice %s..%s: %w", cursor.Format(time.RFC3339), next.Format(time.RFC3339), err)
}

When it happens

Trigger: task_usage_hourly schema migration (which defines the window function) not applied; SIGINT mid-run (ctx.Err()); a conflicting transaction holding row locks on task_usage_hourly past the lock timeout; server restart mid-slice.

Common situations: Running the backfill before `migrate up`; a concurrent writer or the cron entry somehow racing despite lock 4246; long-running slice on a huge month hitting statement_timeout.

Related errors

• stamp watermark: %w
• task_usage_hourly pre-103 hook: %w
• attribution strict-constraint pre-198 hook: %w
• connect to database: %w
• ping database: %w

AI-assisted analysis of multica-ai/multica@2c0912b6ec (2026-08-15). Data as JSON: /api/errors/7b23583168aa4bc0. Report an issue: GitHub.