windmill-labs/windmill · error

Failed to execute test run: ${errorMessage}

Error message

Failed to execute test run: ${errorMessage}

What it means

The test-run tool catches any error raised while launching or awaiting a test execution (queueing the job, building args, polling failures, transport errors) and rethrows it as 'Failed to execute test run: <message>' using formatToolError for a readable cause. The same text is pushed to the tool status so the model can react. It preserves the underlying diagnostic instead of losing it.

Source

Thrown at frontend/src/lib/components/copilot/chat/shared.ts:1829

		// hosts that wire the job hooks) — don't advertise it to in-editor chats.
		if (detachEnabled && config.contextName === 'flow' && !job.success) {
			return (
				summary +
				`\n\nFor per-step statuses and results (subflow steps included), call get_flow_run_details with id="${jobId}".`
			)
		}
		return summary
	} catch (error) {
		// formatToolError, not `error.message`: the generated client puts the server's
		// message in `body` and leaves `message` as the bare status text, so a path
		// runnable aimed at an undeployed flow reported "Not Found" instead of naming
		// the flow it could not find — losing the one diagnostic the run exists for.
		const errorMessage = formatToolError(error)
		config.toolCallbacks.setToolStatus(config.toolId, {
			content: `Test execution failed`,
			error: errorMessage
		})
		throw new Error(`Failed to execute test run: ${errorMessage}`)
	}
}

type FlowStepScriptLoader = (
	moduleValue: { path: string; hash?: string },
	workspace: string
) => Promise<{ content: string; language: ScriptLang }>

type FlowStepPreviewLoader = (path: string, workspace: string) => Promise<FlowValue | undefined>

export type FlowStepTestRunConfig = {
	flowValue: FlowValue
	stepId: string
	args?: Record<string, any> | null
	workspace: string
	toolCallbacks: ToolCallbacks
	toolId: string
	background?: boolean

View on GitHub (pinned to e474e8803c)

Solutions

  1. Read the suffix after 'Failed to execute test run:' for the actual cause and address it (auth, path, network, timeout).
  2. If the cause mentions a missing script/flow, re-open the correct file and retry so the chat context refreshes its path/hash.
  3. Check you are authenticated to the right workspace (re-login or fix the token) if the cause is 401/403.
  4. If the cause is a timeout, see the timeout remedy: optimize the code or check worker availability.
  5. Retry once if the cause is transient (network/5xx); check backend health if it persists.

Example fix

// before
await testRun({ path: 'old/script/path.ts' }) // path deleted
// after
const current = await workspace.getScriptByPath('scripts/current.ts')
await testRun({ path: current.path })
Defensive patterns

Strategy: try-catch

Validate before calling

const path = scriptOptions?.path
if (!path || !(await workspace.scriptExists(path))) {
  throw new Error(`Refusing test run: script ${path} no longer exists`) // avoid deeper failure
}
if (!workspace.isAuthenticated()) await workspace.login()

Type guard

function isTestRunError(e: unknown): e is Error & { message: `Failed to execute test run: ${string}` } {
  return e instanceof Error && e.message.startsWith('Failed to execute test run:')
}

Try / catch

try {
  await executeTestRun(config)
} catch (e) {
  if (isTestRunError(e)) {
    const cause = e.message.slice('Failed to execute test run: '.length)
    // branch on cause: 401 -> re-auth; not found -> refresh path; timeout -> optimize
  }
}

Prevention

When it happens

Trigger: executeTestRun's internal steps threw: the run creation API call failed (network/401/permission), the flow or script referenced by the test could not be found, buildTestRunArgs raised, or the polling helper threw (including the 60s timeout error from the same file).

Common situations: Expired or missing workspace auth so the run API returns 401/403; the script path changed or was deleted since the chat started; backend temporarily unreachable; workspace archived; the underlying cause is the test-timeout error surfacing through this wrapper.

Related errors

AI-assisted analysis of windmill-labs/windmill@e474e8803c (2026-09-03). Data as JSON: /api/errors/3c883f204ccede91. Report an issue: GitHub.