windmill-labs/windmill · error

Timed out after ${Math.round(CASCADE_JOB_TIMEOUT_MS / 60000)

Error message

Timed out after ${Math.round(CASCADE_JOB_TIMEOUT_MS / 60000)}min waiting for job ${jobId} to finish

What it means

The pipeline page polls a cascade job's status every CASCADE_POLL_INTERVAL_MS and gives up after CASCADE_JOB_TIMEOUT_MS minutes. When the deadline elapses without the job reporting completed, the page throws this timeout error so the caller knows the orchestration (run + downstream chain) did not finish, not that the job necessarily failed.

Source

Thrown at frontend/src/routes/(root)/(logged)/pipeline/[folder]/+page.svelte:1576

	// `runDraftAwareCascade`'s finally clears the running-root guard.
	const CASCADE_POLL_INTERVAL_MS = 1000
	const CASCADE_JOB_TIMEOUT_MS = 30 * 60 * 1000
	async function waitJobTerminal(jobId: string): Promise<'success' | 'failure'> {
		const deadline = Date.now() + CASCADE_JOB_TIMEOUT_MS
		while (Date.now() < deadline) {
			try {
				const r = await JobService.getCompletedJobResultMaybe({
					workspace: $workspaceStore!,
					id: jobId,
					getStarted: false
				})
				if (r.completed) return r.success ? 'success' : 'failure'
			} catch {
				// transient — retry on the next tick
			}
			await new Promise((res) => setTimeout(res, CASCADE_POLL_INTERVAL_MS))
		}
		throw new Error(
			`Timed out after ${Math.round(CASCADE_JOB_TIMEOUT_MS / 60000)}min waiting for job ${jobId} to finish`
		)
	}
	// "Run + downstream" over a chain that includes drafts: the backend
	// asset-trigger dispatcher only resolves deployed rows, so the page
	// orchestrates the closure itself (topological order over the resolved
	// graph the user is looking at). Deployed-only chains never come here —
	// they keep the production dispatcher (see onRunProducer).
	async function runDraftAwareCascade(rootPath: string): Promise<string | undefined> {
		if (cascadeRunningRoot) {
			sendUserToast(`A chain run from ${cascadeRunningRoot} is still in progress`, true)
			return undefined
		}
		const closure = computeDownstreamClosure(graphWithDraft, rootPath)
		if (closure.cyclic.length > 0) {
			sendUserToast(
				`Not running ${closure.cyclic.length} script(s) on a dependency cycle: ${closure.cyclic.join(', ')}`,
				true

View on GitHub (pinned to e474e8803c)

Solutions

1. Increase CASCADE_JOB_TIMEOUT_MS in the pipeline page if your cascades legitimately take longer
2. Check the backend job by id for failure/cancellation — the job may be dead rather than slow
3. Verify the backend instance and workers are healthy (job stuck in queue)
4. Re-run the cascade; transient poll failures reset nothing but the job may finish on a second attempt

Example fix

// before
throw new Error(`Timed out after ${Math.round(CASCADE_JOB_TIMEOUT_MS / 60000)}min waiting for job ${jobId} to finish`)
// after
// raise the ceiling for large graphs
const CASCADE_JOB_TIMEOUT_MS = 30 * 60 * 1000 // was 5 * 60 * 1000

const started = Date.now()
const remainingMs = CASCADE_JOB_TIMEOUT_MS - (Date.now() - started)
if (remainingMs <= 0) throw new Error(`Cascade job ${jobId} timed out`)
// also check the job's real status before waiting:
const r = await fetchJob(jobId)
if (r.completed) return r.success ? 'success' : 'failure'

try {
  await waitForCascadeJob(jobId)
} catch (e) {
  if (String(e.message).includes('Timed out after')) {
    const status = await fetchJobStatus(jobId) // inspect real state, maybe still running
    sendUserToast(`Cascade still pending: job ${jobId} (${status})`, true)
  } else throw e
}

When it happens

Trigger: Calling the page's run/downstream cascade orchestration whose job (e.g. asset-trigger dispatcher closure over deployed rows + drafts) takes longer than CASCADE_JOB_TIMEOUT_MS (minutes) to complete, or whose status endpoint keeps throwing transient errors so the poll never observes completion.

Common situations: Very large asset graphs or slow downstream runs exceeding the fixed timeout; backend worker congestion delaying the job; transient network errors hitting the poll endpoint on every tick; a stale jobId whose job was cancelled and will never report completed.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• Timed out waiting for flow ${id} to complete
• Timed out waiting for job ${id}
• Timed out waiting for job ${id} to complete
• Giving up polling job ${jobId} after ${MAX_CONSECUTIVE_POLL_
• Failed to poll dependencies job ${jobId}: ${e?.message ?? e}

AI-assisted analysis of windmill-labs/windmill@e474e8803c (2026-09-03). Data as JSON: /api/errors/6f8c68ff4f122a0a. Report an issue: GitHub.