facebook/flow · critical

To be able to build a thread pool

Error message

To be able to build a thread pool

What it means

ThreadPool::with_thread_count builds a rayon pool with a per-thread stack size — DEFAULT_STACK_SIZE, or $FLOW_STACK_SIZE when that env var is set — and expects the build to succeed (rust_port/crates/flow_utils_concurrency/src/thread_pool.rs:134-137). rayon's build fails when it cannot spawn the requested workers (thread/memory limits) or when the configuration is invalid. Because nearly every Flow entry point constructs this pool at startup, the panic usually appears at process start.

Source

Thrown at rust_port/crates/flow_utils_concurrency/src/thread_pool.rs:137

    }

    pub fn with_thread_count(count: ThreadCount) -> Self {
        #[cfg(target_arch = "wasm32")]
        {
            let _ = count;
            return Self;
        }
        #[cfg(not(target_arch = "wasm32"))]
        {
            let stack_size = Self::stack_size();
            let threads = match count {
                ThreadCount::AllThreads => physical_parallelism(),
                ThreadCount::NumThreads(threads) => threads,
            };
            let builder = rayon::ThreadPoolBuilder::new()
                .stack_size(stack_size)
                .num_threads(threads.get());
            let pool = builder.build().expect("To be able to build a thread pool");
            // Only print the message once
            debug!(
                "Running with {} threads ({} stack size)",
                pool.current_num_threads(),
                human_bytes(stack_size as f64)
            );
            Self(Some(pool))
        }
    }

    pub fn new() -> Self {
        Self::with_thread_count(*THREADS.lock())
    }

    pub fn spawn_many(&self, f: impl Fn() + Sync) {
        #[cfg(target_arch = "wasm32")]
        {
            f();

View on GitHub (pinned to f88ac94bcf)

Solutions

1. Set an explicit modest worker count (--max-workers or [server] max_workers) sized to the container, not the host
2. Review FLOW_STACK_SIZE: it must parse as a number and threads x stack must fit in memory — lower it or raise the memory limit
3. Raise pids.max / ulimit -u / RLIMIT_NOFILE for the process and retry
4. Replace the expect with a fallback that retries with fewer threads, or a clean startup error including threads and stack size

Example fix

# before: host-core-derived workers in a small container with a huge stack
# FLOW_STACK_SIZE=1073741824 flow server --max-workers 64

# after: size both to the container
FLOW_STACK_SIZE=268435456 flow server --max-workers 4

// upstream hardening
let pool = builder.build().unwrap_or_else(|e| {
    panic!("thread pool build failed: {e} (threads={threads}, stack={stack_size})")
});

// Validate thread/stack config before the pool is built
let stack = std::env::var("FLOW_STACK_SIZE").ok().and_then(|s| s.parse::<usize>().ok()).unwrap_or(DEFAULT_STACK_SIZE);
let threads = options.max_workers.max(1) as usize;
if stack.saturating_mul(threads) > available_memory_bytes() {
    eprintln!("threads x stack exceeds memory: {threads} x {stack}");
    std::process::exit(1);
}

fn sane_pool_config(threads: usize, stack: usize) -> bool {
    stack > 0 && stack.saturating_mul(threads) < available_memory_bytes()
}

When it happens

Trigger: num_threads high relative to host limits (large --max-workers, or physical_parallelism reading host cores inside a smaller cgroup); FLOW_STACK_SIZE set so that num_threads x stack_size exceeds available memory; pids.max or RLIMIT_NPROC already exhausted. The sibling panic in stack_size() fires when FLOW_STACK_SIZE does not parse as a number.

Common situations: Containers whose worker count defaults derive from host cores rather than the container quota; operators raising FLOW_STACK_SIZE for deeply recursive inputs without adding memory; CI runners with hard task limits.

Related errors

• $FLOW_STACK_SIZE must be a number, got {s}
• failed to spawn flow_server_main thread
• failed to spawn connect_and_make_request_timed thread
• failed to spawn glean_runner_visit_timeout thread
• failed to spawn flow_cli_main thread

AI-assisted analysis of facebook/flow@f88ac94bcf (2026-08-20). Data as JSON: /api/errors/efd9fe81d20e584e. Report an issue: GitHub.