{"record":{"id":"efd9fe81d20e584e","repo":"facebook/flow","slug":"to-be-able-to-build-a-thread-pool","errorCode":null,"errorMessage":"To be able to build a thread pool","messagePattern":"To be able to build a thread pool","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"rust_port/crates/flow_utils_concurrency/src/thread_pool.rs","lineNumber":137,"sourceCode":"    }\n\n    pub fn with_thread_count(count: ThreadCount) -> Self {\n        #[cfg(target_arch = \"wasm32\")]\n        {\n            let _ = count;\n            return Self;\n        }\n        #[cfg(not(target_arch = \"wasm32\"))]\n        {\n            let stack_size = Self::stack_size();\n            let threads = match count {\n                ThreadCount::AllThreads => physical_parallelism(),\n                ThreadCount::NumThreads(threads) => threads,\n            };\n            let builder = rayon::ThreadPoolBuilder::new()\n                .stack_size(stack_size)\n                .num_threads(threads.get());\n            let pool = builder.build().expect(\"To be able to build a thread pool\");\n            // Only print the message once\n            debug!(\n                \"Running with {} threads ({} stack size)\",\n                pool.current_num_threads(),\n                human_bytes(stack_size as f64)\n            );\n            Self(Some(pool))\n        }\n    }\n\n    pub fn new() -> Self {\n        Self::with_thread_count(*THREADS.lock())\n    }\n\n    pub fn spawn_many(&self, f: impl Fn() + Sync) {\n        #[cfg(target_arch = \"wasm32\")]\n        {\n            f();","sourceCodeStart":119,"sourceCodeEnd":155,"githubUrl":"https://github.com/facebook/flow/blob/f88ac94bcf6992f5d5a158854d94613ebb92c6e6/rust_port/crates/flow_utils_concurrency/src/thread_pool.rs#L119-L155","documentation":"ThreadPool::with_thread_count builds a rayon pool with a per-thread stack size — DEFAULT_STACK_SIZE, or $FLOW_STACK_SIZE when that env var is set — and expects the build to succeed (rust_port/crates/flow_utils_concurrency/src/thread_pool.rs:134-137). rayon's build fails when it cannot spawn the requested workers (thread/memory limits) or when the configuration is invalid. Because nearly every Flow entry point constructs this pool at startup, the panic usually appears at process start.","triggerScenarios":"num_threads high relative to host limits (large --max-workers, or physical_parallelism reading host cores inside a smaller cgroup); FLOW_STACK_SIZE set so that num_threads x stack_size exceeds available memory; pids.max or RLIMIT_NPROC already exhausted. The sibling panic in stack_size() fires when FLOW_STACK_SIZE does not parse as a number.","commonSituations":"Containers whose worker count defaults derive from host cores rather than the container quota; operators raising FLOW_STACK_SIZE for deeply recursive inputs without adding memory; CI runners with hard task limits.","solutions":["Set an explicit modest worker count (--max-workers or [server] max_workers) sized to the container, not the host","Review FLOW_STACK_SIZE: it must parse as a number and threads x stack must fit in memory — lower it or raise the memory limit","Raise pids.max / ulimit -u / RLIMIT_NOFILE for the process and retry","Replace the expect with a fallback that retries with fewer threads, or a clean startup error including threads and stack size"],"exampleFix":"# before: host-core-derived workers in a small container with a huge stack\n# FLOW_STACK_SIZE=1073741824 flow server --max-workers 64\n\n# after: size both to the container\nFLOW_STACK_SIZE=268435456 flow server --max-workers 4\n\n// upstream hardening\nlet pool = builder.build().unwrap_or_else(|e| {\n    panic!(\"thread pool build failed: {e} (threads={threads}, stack={stack_size})\")\n});","handlingStrategy":"validation","validationCode":"// Validate thread/stack config before the pool is built\nlet stack = std::env::var(\"FLOW_STACK_SIZE\").ok().and_then(|s| s.parse::<usize>().ok()).unwrap_or(DEFAULT_STACK_SIZE);\nlet threads = options.max_workers.max(1) as usize;\nif stack.saturating_mul(threads) > available_memory_bytes() {\n    eprintln!(\"threads x stack exceeds memory: {threads} x {stack}\");\n    std::process::exit(1);\n}","typeGuard":"fn sane_pool_config(threads: usize, stack: usize) -> bool {\n    stack > 0 && stack.saturating_mul(threads) < available_memory_bytes()\n}","tryCatchPattern":null,"preventionTips":["Pin worker counts to container quota, not host cores","Keep FLOW_STACK_SIZE numeric and modest; multiply it by workers when budgeting memory","Raise pids/nproc limits when scaling workers up"],"tags":["rayon","thread-pool","resource-limits","stack-size","startup","panic"],"backgroundTag":"thread-pool-build-failed","analyzedSha":"f88ac94bcf6992f5d5a158854d94613ebb92c6e6","analyzedAt":"2026-08-20T10:41:37.992Z","schemaVersion":2},"datasetVersion":"2026-08-23T06:17:17.905Z"}