facebook/flow · critical

failed to dup server->monitor channel

Error message

failed to dup server->monitor channel

What it means

After the monitor spawns the server daemon, it duplicates the daemon's socket endpoint for its own reading: descr_of_in_channel(&server_handle.channels.0).try_clone().expect("failed to dup server->monitor channel") (rust_port/crates/flow_server_monitor/src/flow_server_monitor_server.rs:704-706). TcpStream::try_clone dups the underlying fd/handle; failure means the OS refused the duplication — EMFILE from fd exhaustion is the most common cause, or the daemon died and its channel endpoint was already closed. The expect aborts monitor startup.

Source

Thrown at rust_port/crates/flow_server_monitor/src/flow_server_monitor_server.rs:705

            &init_id,
            _log_file,
            _argv,
            lazy_mode.clone(),
            *no_flowlib,
            *ignore_version,
            file_watcher_pid.map(|p| p as u32),
            start_cause,
            server_options_arc,
            &monitor_options.cli_overrides,
        )
        .unwrap_or_else(|e| panic!("failed to spawn server daemon: {}", e));
        let pid: i32 = server_handle.child.id() as i32;
        // Cross-platform: `TcpStream::try_clone` duplicates the socket on
        // both Unix and Windows. The previous code used
        // `nix::unistd::dup(BorrowedFd)`, which is Unix-only.
        let in_stream = flow_daemon::descr_of_in_channel(&server_handle.channels.0)
            .try_clone()
            .expect("failed to dup server->monitor channel");
        let out_stream = flow_daemon::descr_of_out_channel(&server_handle.channels.1)
            .try_clone()
            .expect("failed to dup monitor->server channel");
        let daemon_handle = Arc::new(Mutex::new(Some(server_handle)));
        let close_daemon_handle = daemon_handle.clone();
        let close = move || {
            let mut guard = match close_daemon_handle.lock() {
                Ok(guard) => guard,
                Err(poisoned) => poisoned.into_inner(),
            };
            if let Some(handle) = guard.as_mut() {
                flow_daemon::close_noerr(handle);
            }
        };

        let server_num = SERVER_NUM.fetch_add(1, Ordering::SeqCst) + 1;
        let name = format!("server #{}", server_num);

View on GitHub (pinned to f88ac94bcf)

Solutions

1. Raise the fd limit (ulimit -n, systemd LimitNOFILE, container nofile limit) and restart the monitor
2. Count open fds over time (ls /proc/<monitor-pid>/fd | wc -l) to find leaks
3. If the daemon died instantly, read its stderr/log — the clone failed because the channel closed, not because of fd limits
4. Upstream: replace the expect with a path that closes the daemon handle and reports the attach failure

Example fix

// before
let in_stream = flow_daemon::descr_of_in_channel(&server_handle.channels.0)
    .try_clone()
    .expect("failed to dup server->monitor channel");

// after
let in_stream = match flow_daemon::descr_of_in_channel(&server_handle.channels.0).try_clone() {
    Ok(s) => s,
    Err(e) => {
        flow_daemon::close_noerr(&mut server_handle);
        panic!("failed to dup server->monitor channel: {e} (fd limit or daemon died?)");
    }
};

// Check fd headroom before the monitor attaches channels
let open = std::fs::read_dir("/proc/self/fd").map(|d| d.count()).unwrap_or(0);
if open + 8 >= fd_limit() {
    eprintln!("monitor: fd headroom too low ({open}/{}); raise ulimit -n", fd_limit());
}

When it happens

Trigger: Monitor startup immediately after spawning the daemon while the monitor process is at RLIMIT_NOFILE (every accepted client and internal channel holds fds), or the daemon crashed on boot so its channels were closed before the clone ran.

Common situations: Long-lived monitors leaking fds; low ulimit -n in service units or containers (default 1024); monitors supervising many servers; daemons failing instantly due to config errors.

Related errors

• failed to dup monitor->server channel
• clone of socket stream for read failed
• Unknown exception reading from the server: {}
• Error sending command to server: {}
• failed to spawn server daemon: {}

AI-assisted analysis of facebook/flow@f88ac94bcf (2026-08-20). Data as JSON: /api/errors/506b623cd4c027f7. Report an issue: GitHub.