{"record":{"id":"7b23583168aa4bc0","repo":"multica-ai/multica","slug":"rollup-slice-s-s-w","errorCode":null,"errorMessage":"rollup slice %s..%s: %w","messagePattern":"rollup slice (.+?)\\.\\.(.+?): %w","errorType":"console","errorClass":null,"httpStatus":null,"severity":"error","filePath":"server/cmd/backfill_task_usage_hourly/main.go","lineNumber":168,"sourceCode":"\tslog.Info(\"backfill range\", \"from\", from.Format(time.RFC3339), \"to\", end.Format(time.RFC3339), \"dry_run\", *dryRun, \"sleep_between_slices\", sleep.String())\n\n\tcursor := from\n\tvar totalRows int64\n\tfor cursor.Before(end) {\n\t\tnext := cursor.AddDate(0, 1, 0)\n\t\tif *dryRun {\n\t\t\tslog.Info(\"would roll up slice\", \"from\", cursor.Format(time.RFC3339), \"to\", next.Format(time.RFC3339))\n\t\t\tcursor = next\n\t\t\tcontinue\n\t\t}\n\t\tvar rows int64\n\t\terr := pool.QueryRow(\n\t\t\tctx,\n\t\t\t`SELECT rollup_task_usage_hourly_window($1::timestamptz, $2::timestamptz)`,\n\t\t\tcursor, next,\n\t\t).Scan(&rows)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"rollup slice %s..%s: %w\", cursor.Format(time.RFC3339), next.Format(time.RFC3339), err)\n\t\t}\n\t\ttotalRows += rows\n\t\tslog.Info(\"rolled up slice\", \"from\", cursor.Format(time.RFC3339), \"to\", next.Format(time.RFC3339), \"rows_touched\", rows)\n\t\tcursor = next\n\t\tif *sleep > 0 && cursor.Before(end) {\n\t\t\tselect {\n\t\t\tcase <-time.After(*sleep):\n\t\t\tcase <-ctx.Done():\n\t\t\t\treturn ctx.Err()\n\t\t\t}\n\t\t}\n\t}\n\n\tif *dryRun {\n\t\tslog.Info(\"dry-run complete; watermark left untouched\")\n\t\treturn nil\n\t}\n\t// Stamp on a fresh context so a SIGINT arriving after the slices","sourceCodeStart":150,"sourceCodeEnd":186,"githubUrl":"https://github.com/multica-ai/multica/blob/2c0912b6ec764b373d44eeea1e80f0d9f11ab417/server/cmd/backfill_task_usage_hourly/main.go#L150-L186","documentation":"Calling the database function rollup_task_usage_hourly_window(cursor, next) for one monthly slice failed. Each slice runs in its own transaction inside the function, so a failure rolls back only that slice and re-running is safe. Causes include the function not existing (schema migration missing), lock contention/timeout on task_usage_hourly rows, statement failure inside the function, or ctx cancelled between slices.","triggerScenarios":"task_usage_hourly schema migration (which defines the window function) not applied; SIGINT mid-run (ctx.Err()); a conflicting transaction holding row locks on task_usage_hourly past the lock timeout; server restart mid-slice.","commonSituations":"Running the backfill before `migrate up`; a concurrent writer or the cron entry somehow racing despite lock 4246; long-running slice on a huge month hitting statement_timeout.","solutions":["Verify the function exists: \\df rollup_task_usage_hourly_window — if not, run migrations first","If the wrapped error is context.Canceled, that is a clean SIGINT stop: just re-run; it resumes from the watermark/already-rolled-up slices","Raise statement_timeout / lock_timeout for the session, or run with -sleep to throttle slice pressure","Check pg_stat_activity for blockers on task_usage_hourly and resolve them"],"exampleFix":"# before\nbackfill_task_usage_hourly -db $DSN   # fails mid-slice on timeout\n\n# after\npsql $DSN -c \"SET statement_timeout='0'\"  # or run with throttling:\nbackfill_task_usage_hourly -db $DSN -sleep 500ms","handlingStrategy":"retry","validationCode":"var fnExists bool\npool.QueryRow(ctx, `SELECT to_regprocedure('rollup_task_usage_hourly_window(timestamptz,timestamptz)') IS NOT NULL`).Scan(&fnExists)","typeGuard":null,"tryCatchPattern":"if err != nil {\n    return fmt.Errorf(\"rollup slice %s..%s: %w\", cursor.Format(time.RFC3339), next.Format(time.RFC3339), err)\n}","preventionTips":["Apply the task_usage_hourly migration before backfilling","Run with -sleep between slices to reduce lock pressure","Interrupted runs are safe: each slice is its own transaction, just re-run"],"tags":["go","postgres","backfill","sql-function","timeout"],"backgroundTag":null,"analyzedSha":"2c0912b6ec764b373d44eeea1e80f0d9f11ab417","analyzedAt":"2026-08-15T13:25:18.241Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}