{"record":{"id":"a61ab89946c67de0","repo":"zeroclaw-labs/zeroclaw","slug":"lucid-command-timed-out-after-ms-failed-to-term","errorCode":null,"errorMessage":"lucid command timed out after {}ms; failed to terminate and reap child: {cleanup_error}","messagePattern":"lucid command timed out after (.+?)ms; failed to terminate and reap child: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"crates/zeroclaw-memory/src/lucid.rs","lineNumber":298,"sourceCode":"        .await;\n\n        let status = match output {\n            Ok(status) => status?,\n            Err(_) => {\n                let cleanup_error = child.kill().await.err().map(|error| error.to_string());\n                ::zeroclaw_log::record!(\n                    ERROR,\n                    ::zeroclaw_log::Event::new(module_path!(), ::zeroclaw_log::Action::Timeout)\n                        .with_outcome(::zeroclaw_log::EventOutcome::Failure)\n                        .with_attrs(::serde_json::json!({\n                            \"command\": lucid_cmd,\n                            \"timeout_ms\": timeout_window.as_millis() as u64,\n                            \"cleanup_error\": cleanup_error,\n                        })),\n                    \"lucid command timed out\"\n                );\n                if let Some(cleanup_error) = cleanup_error {\n                    anyhow::bail!(\n                        \"lucid command timed out after {}ms; failed to terminate and reap child: {cleanup_error}\",\n                        timeout_window.as_millis()\n                    );\n                }\n                anyhow::bail!(\n                    \"lucid command timed out after {}ms\",\n                    timeout_window.as_millis()\n                );\n            }\n        };\n\n        if !status.success() {\n            let stderr = String::from_utf8_lossy(&stderr_bytes);\n            anyhow::bail!(\"lucid command failed: {stderr}\");\n        }\n\n        Ok(String::from_utf8_lossy(&stdout_bytes).to_string())\n    }","sourceCodeStart":280,"sourceCodeEnd":316,"githubUrl":"https://github.com/zeroclaw-labs/zeroclaw/blob/88bb9c8533fc57ed7a03e36ca7c9ed2bf8336dcc/crates/zeroclaw-memory/src/lucid.rs#L280-L316","documentation":"A lucid CLI subprocess exceeded its timeout window (recall_timeout_ms / store_timeout_ms, default or [storage.lucid.<alias>] override), and the subsequent terminate-and-reap cleanup also failed. The message reports both the elapsed window and the cleanup error because a failed reap means the child may still be running or become a zombie — that is what distinguishes it from the plain timeout variant.","triggerScenarios":"A lucid command hangs (remote endpoint stalls, binary waits on stdin or a stuck lock) past the configured timeout, then killing/waiting on the child errors — e.g. the process exits concurrently with the signal, or OS-level reaping races.","commonSituations":"Aggressively low lucid timeouts on slow networks; half-open connections to the lucid service; heavy load making SIGTERM delivery race with process exit; containers with restricted process-management privileges.","solutions":["Raise the timeout: set `recall_timeout_ms` / `store_timeout_ms` under `[storage.lucid.<alias>]` above observed lucid p99 latency.","Check for leftover lucid processes after this error (ps aux | grep lucid) and kill them manually — reaping already failed.","Verify the lucid binary and endpoint are healthy by running the same lucid command manually with the same environment."],"exampleFix":"# before\n[storage.lucid.main]\nrecall_timeout_ms = 2000\n\n# after\n[storage.lucid.main]\nrecall_timeout_ms = 15000\nstore_timeout_ms = 20000","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"let entries = match lucid_recall(query, limit, None, since, until).await {\n    Ok(v) => v,\n    Err(e) if e.to_string().contains(\"lucid command timed out\") => {\n        // local sqlite results are still valid: degrade and alert\n        // (this variant also means reaping failed — sweep for orphaned\n        // lucid processes before continuing)\n        sweep_orphaned_lucid_processes();\n        local.recall(query, limit, None, since, until).await?\n    }\n    Err(e) => return Err(e),\n};","preventionTips":["Size recall_timeout_ms/store_timeout_ms to the observed p99 of the lucid binary plus network, not the average.","Monitor timeout rate; a rising rate is an early signal of lucid endpoint degradation.","Run lucid close to the service (same host/network) to keep the timeout headroom meaningful."],"tags":["lucid","timeout","subprocess","process-cleanup","zombie"],"backgroundTag":"subprocess-timeout","analyzedSha":"88bb9c8533fc57ed7a03e36ca7c9ed2bf8336dcc","analyzedAt":"2026-08-23T01:07:41.857Z","schemaVersion":2},"datasetVersion":"2026-08-23T08:06:27.607Z"}