What causes "Failed to execute test run: ${errorMessage}" in windmill-labs/windmill?

Read the suffix after 'Failed to execute test run:' for the actual cause and address it (auth, path, network, timeout).

Failed to execute test run

Error message

Failed to execute test run: ${errorMessage}

What it means

The test-run tool catches any error raised while launching or awaiting a test execution (queueing the job, building args, polling failures, transport errors) and rethrows it as 'Failed to execute test run: <message>' using formatToolError for a readable cause. The same text is pushed to the tool status so the model can react. It preserves the underlying diagnostic instead of losing it.

Solutions

Read the suffix after 'Failed to execute test run:' for the actual cause and address it (auth, path, network, timeout).
If the cause mentions a missing script/flow, re-open the correct file and retry so the chat context refreshes its path/hash.
Check you are authenticated to the right workspace (re-login or fix the token) if the cause is 401/403.
If the cause is a timeout, see the timeout remedy: optimize the code or check worker availability.
Retry once if the cause is transient (network/5xx); check backend health if it persists.

Example fix

// before
await testRun({ path: 'old/script/path.ts' }) // path deleted
// after
const current = await workspace.getScriptByPath('scripts/current.ts')
await testRun({ path: current.path })

Defensive patterns

Strategy: try-catch

Validate before calling

const path = scriptOptions?.path
if (!path || !(await workspace.scriptExists(path))) {
  throw new Error(`Refusing test run: script ${path} no longer exists`) // avoid deeper failure
}
if (!workspace.isAuthenticated()) await workspace.login()

Type guard

function isTestRunError(e: unknown): e is Error & { message: `Failed to execute test run: ${string}` } {
  return e instanceof Error && e.message.startsWith('Failed to execute test run:')
}

Try / catch

try {
  await executeTestRun(config)
} catch (e) {
  if (isTestRunError(e)) {
    const cause = e.message.slice('Failed to execute test run: '.length)
    // branch on cause: 401 -> re-auth; not found -> refresh path; timeout -> optimize
  }
}

Prevention

Confirm workspace auth is valid before starting test runs.
Re-open the script/flow after renames so the chat context holds the current path/hash.
Check backend reachability/health when running against a local dev instance.
Treat the embedded cause message as the primary diagnostic and route on it.

When it happens

Trigger: executeTestRun's internal steps threw: the run creation API call failed (network/401/permission), the flow or script referenced by the test could not be found, buildTestRunArgs raised, or the polling helper threw (including the 60s timeout error from the same file).

Common situations: Expired or missing workspace auth so the run API returns 401/403; the script path changed or was deleted since the chat started; backend temporarily unreachable; workspace archived; the underlying cause is the test-timeout error surfacing through this wrapper.

Related errors

AI-assisted analysis of windmill-labs/windmill@e474e8803c (2026-09-03). Data as JSON: /api/errors/3c883f204ccede91. Report an issue: GitHub.

Appendix: source

Thrown at frontend/src/lib/components/copilot/chat/shared.ts:1829

		// hosts that wire the job hooks) — don't advertise it to in-editor chats.
		if (detachEnabled && config.contextName === 'flow' && !job.success) {
			return (
				summary +
				`\n\nFor per-step statuses and results (subflow steps included), call get_flow_run_details with id="${jobId}".`
			)
		}
		return summary
	} catch (error) {
		// formatToolError, not `error.message`: the generated client puts the server's
		// message in `body` and leaves `message` as the bare status text, so a path
		// runnable aimed at an undeployed flow reported "Not Found" instead of naming
		// the flow it could not find — losing the one diagnostic the run exists for.
		const errorMessage = formatToolError(error)
		config.toolCallbacks.setToolStatus(config.toolId, {
			content: `Test execution failed`,
			error: errorMessage
		})
		throw new Error(`Failed to execute test run: ${errorMessage}`)
	}
}

type FlowStepScriptLoader = (
	moduleValue: { path: string; hash?: string },
	workspace: string
) => Promise<{ content: string; language: ScriptLang }>

type FlowStepPreviewLoader = (path: string, workspace: string) => Promise<FlowValue | undefined>

export type FlowStepTestRunConfig = {
	flowValue: FlowValue
	stepId: string
	args?: Record<string, any> | null
	workspace: string
	toolCallbacks: ToolCallbacks
	toolId: string
	background?: boolean

View on GitHub (pinned to e474e8803c)