facebook/flow · critical

failed to dup server->monitor channel

Error message

failed to dup server->monitor channel

What it means

After the monitor spawns the server daemon, it duplicates the daemon's socket endpoint for its own reading: descr_of_in_channel(&server_handle.channels.0).try_clone().expect("failed to dup server->monitor channel") (rust_port/crates/flow_server_monitor/src/flow_server_monitor_server.rs:704-706). TcpStream::try_clone dups the underlying fd/handle; failure means the OS refused the duplication — EMFILE from fd exhaustion is the most common cause, or the daemon died and its channel endpoint was already closed. The expect aborts monitor startup.

Solutions

1. Raise the fd limit (ulimit -n, systemd LimitNOFILE, container nofile limit) and restart the monitor
2. Count open fds over time (ls /proc/<monitor-pid>/fd | wc -l) to find leaks
3. If the daemon died instantly, read its stderr/log — the clone failed because the channel closed, not because of fd limits
4. Upstream: replace the expect with a path that closes the daemon handle and reports the attach failure

Example fix

// before
let in_stream = flow_daemon::descr_of_in_channel(&server_handle.channels.0)
    .try_clone()
    .expect("failed to dup server->monitor channel");

// after
let in_stream = match flow_daemon::descr_of_in_channel(&server_handle.channels.0).try_clone() {
    Ok(s) => s,
    Err(e) => {
        flow_daemon::close_noerr(&mut server_handle);
        panic!("failed to dup server->monitor channel: {e} (fd limit or daemon died?)");
    }
};

// Check fd headroom before the monitor attaches channels
let open = std::fs::read_dir("/proc/self/fd").map(|d| d.count()).unwrap_or(0);
if open + 8 >= fd_limit() {
    eprintln!("monitor: fd headroom too low ({open}/{}); raise ulimit -n", fd_limit());
}

When it happens

Trigger: Monitor startup immediately after spawning the daemon while the monitor process is at RLIMIT_NOFILE (every accepted client and internal channel holds fds), or the daemon crashed on boot so its channels were closed before the clone ran.

Common situations: Long-lived monitors leaking fds; low ulimit -n in service units or containers (default 1024); monitors supervising many servers; daemons failing instantly due to config errors.

Related errors

• failed to dup monitor->server channel
• clone of socket stream for read failed
• Error sending command to server
• failed to spawn LSP log flusher
• failed to spawn server daemon

AI-assisted analysis of facebook/flow@5c86586199 (2026-08-20). Data as JSON: /api/errors/506b623cd4c027f7. Report an issue: GitHub.

Appendix: source

Thrown at rust_port/crates/flow_server_monitor/src/flow_server_monitor_server.rs:725

            &init_id,
            _log_file,
            _argv,
            lazy_mode.clone(),
            *no_flowlib,
            *ignore_version,
            file_watcher_pid.map(|p| p as u32),
            start_cause,
            server_options_arc,
            &monitor_options.cli_overrides,
        )
        .unwrap_or_else(|e| panic!("failed to spawn server daemon: {}", e));
        let pid: i32 = server_handle.child.id() as i32;
        // Cross-platform: `TcpStream::try_clone` duplicates the socket on
        // both Unix and Windows. The previous code used
        // `nix::unistd::dup(BorrowedFd)`, which is Unix-only.
        let in_stream = flow_daemon::descr_of_in_channel(&server_handle.channels.0)
            .try_clone()
            .expect("failed to dup server->monitor channel");
        let out_stream = flow_daemon::descr_of_out_channel(&server_handle.channels.1)
            .try_clone()
            .expect("failed to dup monitor->server channel");
        let daemon_handle = Arc::new(Mutex::new(Some(server_handle)));
        let close_daemon_handle = daemon_handle.clone();
        let close = move || {
            let mut guard = match close_daemon_handle.lock() {
                Ok(guard) => guard,
                Err(poisoned) => poisoned.into_inner(),
            };
            if let Some(handle) = guard.as_mut() {
                flow_daemon::close_noerr(handle);
            }
        };

        let server_num = SERVER_NUM.fetch_add(1, Ordering::SeqCst) + 1;
        let name = format!("server #{}", server_num);

View on GitHub (pinned to 5c86586199)