dbt-labs/dbt-core · error

failed to spawn worker thread

Error message

failed to spawn worker thread

What it means

`fetch_catalog_data` spawns rayon/worker threads to query catalog metadata in parallel and panics if `std::thread::Builder` (or equivalent spawn) fails. Thread spawn fails essentially only on OS resource exhaustion — inability to allocate a stack or hit the process/thread limit.

Solutions

1. Check `ulimit -u` and container pids/cgroup limits and raise them
2. Reduce memory pressure or the per-thread stack size (FS_DEFAULT_STACK_SIZE-derived setting)
3. Lower catalog query concurrency/batch thread count
4. Retry on a larger machine / rerun after other runaway processes exit

Example fix

// before
let handle = thread::Builder::new().stack_size(...).spawn(...).expect("failed to spawn worker thread");
// after
let handle = thread::Builder::new().stack_size(...).spawn(...)
    .map_err(|e| ErrFatal.load_error(format!("failed to spawn catalog worker thread: {e}")))?;

// pre-check thread headroom
let nproc = std::thread::available_parallelism().map(|n| n.get()).unwrap_or(1);
if ulimit_u() < nproc * 2 { eprintln!("thread limit too low for catalog workers"); }

match thread::Builder::new().stack_size(sz).spawn(work) {
    Ok(h) => handles.push(h),
    Err(e) => errors.push(format!("catalog worker spawn failed: {e}")),
}

When it happens

Trigger: `try_fetch_catalog` → `write_catalog_json` → `fetch_catalog_data` calls `.spawn()` with a large configured stack size (`thread_stack_size`) on a memory-constrained machine, or the process hits the OS thread/ulimit cap (ulimit -u, cgroup pids.max, containers with low thread limits).

Common situations: Running dbt docs generate in constrained Docker/K8s containers; large `DBT_*` stack-size env settings multiplying memory per worker; hitting `nproc`/pid limits with many catalog batches.

Related errors

• OS can't spawn worker thread

AI-assisted analysis of dbt-labs/dbt-core@0267ce9170 (2026-09-07). Data as JSON: /api/errors/25631fc5ad0d959c. Report an issue: GitHub.

Appendix: source

Thrown at crates/dbt-main/src/dbt_lib.rs:2987

                            Ok(record_batch) => {
                                shared_results_clone.lock().unwrap().push(record_batch);
                            }
                            Err(e) => {
                                let msg = format!("[Non-critical] Issue processing catalog for schema '{database}.{schema}': {e}");
                                emit_info_log_message(&msg);
                                shared_errors_clone.lock().unwrap().push(msg);
                            }
                        },
                        Err(e) => {
                            let msg = format!("[Non-critical] Issue fetching catalog for schema '{database}.{schema}': {e}");
                            emit_info_log_message(&msg);
                            shared_errors_clone.lock().unwrap().push(msg);
                        }
                    }
                }
                Ok(())
            })
            .expect("failed to spawn worker thread");
        handles.push(handle);
    }

    // Do this so that handles are not abandoned immediately while we poll
    let _handles = handles;

    // Do not await workers directly as they may lock due to ADBC issues
    loop {
        tokio::time::sleep(POLL_INTERVAL).await;

        let tracker_snapshot = progress_tracker.lock().unwrap().clone();

        // All workers finished normally
        if tracker_snapshot.is_empty() {
            emit_info_log_message("Fetched full catalog.json results");
            break;
        }

View on GitHub (pinned to 0267ce9170)