Hmbown/CodeWhale · error

inline stderr logger panicked

Error message

inline stderr logger panicked

What it means

The stderr counterpart of the inline `start` logger panic: after the inline command exits, joining the stderr pump thread yields Err because the thread panicked, and this error is raised. The child's stderr may be missing from the lane log and lane completion (`lane_completed` event) is not recorded when logging failed.

Solutions

  1. Locate the panic site from the thread's printed panic message and fix the underlying I/O condition in `stream_child_output`.
  2. Keep `record.log_path` and its directory present and writable until all logger threads have joined.
  3. Check disk space and fd limits for long-running or chatty commands.
  4. Refactor `stream_child_output` to return Result instead of unwrapping so join() surfaces the true error and it can be recorded via `append_log_event`.
  5. Re-run the lane and confirm the `lane_completed` event lands in the log after a clean run.

Example fix

// before
file.write_all(chunk).expect("stderr logger");
// after
file.write_all(chunk).context("stderr log append")?;
Defensive patterns

Strategy: try-catch

Validate before calling

let Ok(meta) = std::fs::metadata(&record.log_path) else { /* create it */ };
assert!(!meta.permissions().readonly(), "lane log is read-only");

Type guard

fn is_panic_join<T>(r: &std::thread::Result<T>) -> bool { r.is_err() }

Try / catch

match lane_start_inline(spec) {
    Err(e) if e.to_string().contains("inline stderr logger panicked") => {
        eprintln!("stderr logger thread panicked: {e:#}; lane_completed not recorded, inspect log dir");
    }
    Err(e) => eprintln!("start failed: {e:#}"),
    Ok(record) => { /* normal path */ }
}

Prevention

When it happens

Trigger: Inline lane `start` where the stderr thread (`stream_child_output(stderr, stderr_log, "stderr")`) panics — most commonly an unwrap on a failed append to `record.log_path` triggered by the command writing to stderr at the wrong moment (e.g. after cleanup started).

Common situations: Log handle invalidated mid-run (directory removed, permissions, disk full); panics inside `stream_child_output` on serialization of unexpected stderr content; stop/attach flows closing resources while the logger thread still writes.

Related errors


AI-assisted analysis of Hmbown/CodeWhale@73e0f67d83 (2026-09-22). Data as JSON: /api/errors/9d088ec5d13ce46d. Report an issue: GitHub.

Appendix: source

Thrown at crates/lane/src/runtime.rs:1021

        let stderr = child
            .stderr
            .take()
            .context("inline lane child stderr was not piped")?;
        let stdout_log = record.log_path.clone();
        let stderr_log = record.log_path.clone();
        let stdout_thread =
            thread::spawn(move || stream_child_output(stdout, stdout_log, "stdout"));
        let stderr_thread =
            thread::spawn(move || stream_child_output(stderr, stderr_log, "stderr"));
        let status = child
            .wait()
            .with_context(|| format!("wait for inline command {:?}", spec.command))?;
        let stdout_result = stdout_thread
            .join()
            .map_err(|_| anyhow::anyhow!("inline stdout logger panicked"))?;
        let stderr_result = stderr_thread
            .join()
            .map_err(|_| anyhow::anyhow!("inline stderr logger panicked"))?;
        let logging_error = stdout_result.err().or_else(|| stderr_result.err());

        if status.success() && logging_error.is_none() {
            append_log_event(
                &record.log_path,
                serde_json::json!({"type": "lane_completed", "lane_id": record.id}),
            )?;
            let _ = registry.mark_terminal_if_active(record, LaneStatus::Completed)?;
        } else {
            append_log_event(
                &record.log_path,
                serde_json::json!({
                    "type": "lane_failed",
                    "lane_id": record.id,
                    "status": format!("{status}"),
                    "logging_error": logging_error.as_ref().map(ToString::to_string),
                }),
            )?;

View on GitHub (pinned to 73e0f67d83)