FuelLabs/fuel-core · critical

Failed to create the local runner

Error message

Failed to create the local runner

What it means

In `ImporterInner::run`, `LocalRunner::new()` builds a dedicated 2-thread rayon thread pool used for block execution and serialization. The result is unwrapped with `.expect("Failed to create the local runner")`, so this message is a panic, not a returned error. It fires only when rayon's `ThreadPoolBuilder::build()` fails, which is practically limited to thread-creation failures (e.g. resource limits) — rayon only errors if its internal spawn fails.

Source

Thrown at crates/services/importer/src/importer.rs:494

struct VerifyAndExecutionResult {
    tx_status: Vec<TransactionExecutionStatus>,
    events: Vec<Event>,
    changes: Changes,
}

struct PrepareImportResult {
    result: UncommittedResult<Changes>,
    block_changes: Changes,
}

impl<IDatabase, E, V> ImporterInner<IDatabase, E, V>
where
    IDatabase: ImporterDatabase + Transactional,
    E: Validator,
    V: BlockVerifier,
{
    async fn run(&mut self) {
        let local_runner = LocalRunner::new().expect("Failed to create the local runner");
        while let Some(command) = self.commands.recv().await {
            match command {
                Commands::Stop => break,
                Commands::CommitResult {
                    result,
                    permit,
                    callback,
                } => {
                    let result = self.commit_result(&local_runner, permit, result);
                    let _ = callback.send(result);
                }
                #[cfg(test)]
                Commands::VerifyAndExecuteBlock {
                    sealed_block,
                    callback,
                } => {
                    let result =
                        self.verify_and_execute_block(&local_runner, sealed_block);

View on GitHub (pinned to add100d30d)

Solutions

1. Raise the process/thread limit: `ulimit -u` on Linux or the container's pids cgroup limit (`--pids-limit` in Docker, `pids.max` in Kubernetes).
2. Reduce overall thread pressure: lower rayon global thread counts (`RAYON_NUM_THREADS`) and other concurrency settings so thread creation succeeds.
3. Check host memory — each thread needs stack space; free memory or lower stack sizes if the failure is OOM at spawn.
4. If failures persist, replace the `.expect` with propagated error handling so the importer reports the root `rayon` error instead of panicking.

Example fix

// before
let local_runner = LocalRunner::new().expect("Failed to create the local runner");
// after
let local_runner = LocalRunner::new().map_err(|e| {
    anyhow::anyhow!("Failed to create the local runner (check thread/memory limits): {e:#}")
})?;

// verify the process can spawn threads before starting the importer
let probe = std::thread::Builder::new().stack_size(64 * 1024).spawn(|| {});
if probe.is_err() {
    return Err(anyhow!("Cannot create threads (check ulimit -u / pids cgroup limit / memory)"));
}

fn can_create_rayon_pool(threads: usize) -> bool {
    rayon::ThreadPoolBuilder::new().num_threads(threads).build().is_ok()
}

// the source panics via expect, so defend before calling run()
if !can_create_rayon_pool(2) {
    return Err(anyhow!("Failed to create the local runner: thread pool unavailable"));
}
importer.run().await;

When it happens

Trigger: `Importer::run` starts (importer.rs:494) and `rayon::ThreadPoolBuilder::new().num_threads(2).build()` fails, typically because the OS refused to spawn threads (hit `RLIMIT_NPROC`/`ulimit -u`, cgroup pids limit, or memory exhaustion for thread stacks).

Common situations: Running fuel-core in restricted containers/Kubernetes with low `pids.max` or `ulimit -u`; deeply nested rayon pools in environments that disallow thread creation; heavily loaded hosts at the process/thread limit.

Related errors

• epoch token lock poisoned: {}
• Not implemented yet
• The lock is poisoned: {}
• cannot access epoch token, poisoned lock

AI-assisted analysis of FuelLabs/fuel-core@add100d30d (2026-09-05). Data as JSON: /api/errors/bc5bcb311d24c935. Report an issue: GitHub.