What causes "isolate thread panicked: {e}" in windmill-labs/windmill?

Find the original panic message/backtrace in the worker logs (it accompanies this JoinError)

isolate thread panicked: {e}

Error message

isolate thread panicked: {e}

What it means

After the result channel resolves, ExecutingIsolate::wait awaits the isolate thread's JoinHandle. If the thread panicked, the JoinError is wrapped into this error, indicating the dedicated Deno isolate crashed while running (or finishing) the script.

Source

Thrown at backend/windmill-runtime-nativets/src/dedicated.rs:28

pub struct PrewarmedResult {
    pub result: Result<Box<RawValue>, String>,
    pub logs: String,
}

pub struct ExecutingIsolate {
    result_rx: tokio::sync::oneshot::Receiver<PrewarmedResult>,
    handle: tokio::task::JoinHandle<anyhow::Result<()>>,
}

impl ExecutingIsolate {
    pub async fn wait(self) -> anyhow::Result<PrewarmedResult> {
        let result = self
            .result_rx
            .await
            .map_err(|_| anyhow::anyhow!("isolate result channel closed"))?;
        self.handle
            .await
            .map_err(|e| anyhow::anyhow!("isolate thread panicked: {e}"))??;
        Ok(result)
    }
}

pub struct PrewarmedIsolate {
    args_tx: Option<tokio::sync::oneshot::Sender<String>>,
    result_rx: Option<tokio::sync::oneshot::Receiver<PrewarmedResult>>,
    ready_rx: Option<tokio::sync::oneshot::Receiver<()>>,
    handle: Option<tokio::task::JoinHandle<anyhow::Result<()>>>,
}

/// Parse a JSON args object and reorder into positional args matching `arg_names`.
fn args_to_positional(args_json: &str, arg_names: &[String]) -> Vec<Option<Box<RawValue>>> {
    let map: HashMap<String, Box<RawValue>> = serde_json::from_str(args_json).unwrap_or_default();
    arg_names
        .iter()
        .map(|name| map.get(name).cloned())
        .collect()

View on GitHub (pinned to e474e8803c)

Solutions

Find the original panic message/backtrace in the worker logs (it accompanies this JoinError)
Reduce script memory usage or raise the worker's memory limits
Reduce concurrent isolates to rule out resource contention
Update the backend/runtime crates; deno_core panics are fixed regularly
File a bug with the panic backtrace and the triggering script if persistent

Defensive patterns

Strategy: retry

Try / catch

match exec.wait().await {
    Err(e) if e.to_string().starts_with("isolate thread panicked") => {
        log::error!("isolate crashed, restarting worker pool: {e}");
        pool.restart().await?;
        pool.run(args).await
    }
    other => other,
}

Prevention

Reduce isolate concurrency to stay within worker memory
Upgrade the backend regularly — deno_core panics get patched
Capture and attach the thread panic backtrace from logs when filing issues
Stress-test heavy TS scripts before deploying them to production workers

When it happens

Trigger: The spawned thread running the Deno isolate panics during execution — a runtime bug in the isolate pool, memory corruption/limits, or a panic inside the module-load/execution plumbing.

Common situations: Heavy TS scripts exhausting isolate memory; bugs in vendored deno_core versions; native module crashes inside the isolate; workers under extreme concurrency.

Related errors

AI-assisted analysis of windmill-labs/windmill@e474e8803c (2026-09-03). Data as JSON: /api/errors/0fd27826a2f2e3eb. Report an issue: GitHub.