What causes "creating threadpool failed" in atuinsh/atuin?

Raise thread/process limits: docker --pids-limit, systemd TasksMax, ulimit -u

creating threadpool failed

Error message

creating threadpool failed

What it means

atuin-nucleo (Atuin's nucleo fork) builds its matcher worker pool with rayon, sized by the worker_threads argument or std::thread::available_parallelism. rayon's ThreadPoolBuilder::build returns Err when worker threads cannot be spawned — thread/process limits or memory exhaustion — and Worker::new expects the Result, panicking with 'creating threadpool failed' when search starts.

Source

Thrown at crates/atuin-nucleo/src/worker.rs:81

    }

    pub(crate) fn set_scorer(&mut self, scorer: Option<Scorer<T>>) {
        self.scorer = scorer;
    }

    pub(crate) fn new(
        worker_threads: Option<usize>,
        config: Config,
        notify: Arc<dyn Fn() + Sync + Send>,
        cols: u32,
    ) -> (ThreadPool, Self) {
        let worker_threads = worker_threads
            .unwrap_or_else(|| std::thread::available_parallelism().map_or(4, |it| it.get()));
        let pool = rayon::ThreadPoolBuilder::new()
            .thread_name(|i| format!("nucleo worker {i}"))
            .num_threads(worker_threads)
            .build()
            .expect("creating threadpool failed");
        let matchers = (0..worker_threads)
            .map(|_| UnsafeCell::new(atuin_nucleo_matcher::Matcher::new(config.clone())))
            .collect();
        let worker = Worker {
            running: false,
            matchers: Matchers(matchers),
            last_snapshot: 0,
            matches: Vec::new(),
            // just a placeholder
            pattern: MultiPattern::new(cols as usize),
            sort_results: true,
            reverse_items: false,
            canceled: Arc::new(AtomicBool::new(false)),
            should_notify: Arc::new(AtomicBool::new(false)),
            was_canceled: false,
            notify,
            items: Arc::new(boxcar::Vec::with_capacity(2 * 1024, cols)),
            in_flight: Vec::with_capacity(64),

View on GitHub (pinned to 202f6ad98e)

Solutions

Raise thread/process limits: docker --pids-limit, systemd TasksMax, ulimit -u
Reduce parallelism: configure fewer matcher worker threads if your embedding exposes the knob
Free memory or lower thread stack size so rayon workers can spawn

Example fix

# before
docker run --pids-limit 64 ...

# after
docker run --pids-limit 512 ...

Defensive patterns

Strategy: validation

Validate before calling

fn can_spawn_threads(n: usize) -> bool {
    let handles: Vec<_> = (0..n).map(|_| std::thread::spawn(|| ())).collect();
    handles.into_iter().all(|h| h.join().is_ok())
}

let desired = std::thread::available_parallelism().map(|n| n.get()).unwrap_or(1);
assert!(can_spawn_threads(desired), "thread budget too low for the matcher pool");

Try / catch

let engine = std::panic::catch_unwind(|| {
    atuin_nucleo::Nucleo::<atuin_history::History>::new(config, &notify, cols)
});
// on Err: retry with a smaller worker budget or surface a clear resource error

Prevention

Set generous pids limits (--pids-limit, TasksMax, ulimit -u) where search runs
Cap matcher worker threads explicitly in constrained environments
Check memory headroom: each rayon worker maps a thread stack

When it happens

Trigger: Constructing the matcher (atuin search / interactive TUI startup) when thread creation fails: cgroup pids.max reached, RLIMIT_NPROC (ulimit -u) exhausted, systemd TasksMax hit, or out of memory to map thread stacks. Note rayon treats num_threads=0 as 'use default', so Some(0) is not the trigger.

Common situations: Containers with low --pids-limit; shared machines where the user hits nproc; memory-constrained environments where default 2MB thread stacks cannot be allocated.

Related errors

AI-assisted analysis of atuinsh/atuin@202f6ad98e (2026-08-16). Data as JSON: /api/errors/6fc63a5a01b242d9. Report an issue: GitHub.