What causes "failed to dup monitor->server channel" in facebook/flow?

Raise the fd limit (ulimit -n / LimitNOFILE / container nofile) — being one fd short is the signature of this exact panic

facebook/flow · critical

failed to dup monitor->server channel

Error message

failed to dup monitor->server channel

What it means

The second channel duplication at monitor startup: descr_of_out_channel(&server_handle.channels.1).try_clone().expect("failed to dup monitor->server channel") (rust_port/crates/flow_server_monitor/src/flow_server_monitor_server.rs:707-708). The out-channel (monitor->server direction) must also be duplicated so the monitor holds independent read/write handles. The failure modes are identical to the in-channel dup: EMFILE at the fd ceiling, or the daemon endpoint already closed.

Source

Thrown at rust_port/crates/flow_server_monitor/src/flow_server_monitor_server.rs:708

            lazy_mode.clone(),
            *no_flowlib,
            *ignore_version,
            file_watcher_pid.map(|p| p as u32),
            start_cause,
            server_options_arc,
            &monitor_options.cli_overrides,
        )
        .unwrap_or_else(|e| panic!("failed to spawn server daemon: {}", e));
        let pid: i32 = server_handle.child.id() as i32;
        // Cross-platform: `TcpStream::try_clone` duplicates the socket on
        // both Unix and Windows. The previous code used
        // `nix::unistd::dup(BorrowedFd)`, which is Unix-only.
        let in_stream = flow_daemon::descr_of_in_channel(&server_handle.channels.0)
            .try_clone()
            .expect("failed to dup server->monitor channel");
        let out_stream = flow_daemon::descr_of_out_channel(&server_handle.channels.1)
            .try_clone()
            .expect("failed to dup monitor->server channel");
        let daemon_handle = Arc::new(Mutex::new(Some(server_handle)));
        let close_daemon_handle = daemon_handle.clone();
        let close = move || {
            let mut guard = match close_daemon_handle.lock() {
                Ok(guard) => guard,
                Err(poisoned) => poisoned.into_inner(),
            };
            if let Some(handle) = guard.as_mut() {
                flow_daemon::close_noerr(handle);
            }
        };

        let server_num = SERVER_NUM.fetch_add(1, Ordering::SeqCst) + 1;
        let name = format!("server #{}", server_num);

        let (start_fn, connection) =
            ServerConnection::create(name.clone(), in_stream, out_stream, close, handle_response);

View on GitHub (pinned to f88ac94bcf)

Solutions

Raise the fd limit (ulimit -n / LimitNOFILE / container nofile) — being one fd short is the signature of this exact panic
Restart the monitor after closing leaked fds; verify with ls /proc/<monitor-pid>/fd | wc -l
Check the daemon's boot log if the channel was closed rather than the clone refused
Upstream: pair both clones with a cleanup path that closes the daemon handle on either failure

Example fix

// before
let out_stream = flow_daemon::descr_of_out_channel(&server_handle.channels.1)
    .try_clone()
    .expect("failed to dup monitor->server channel");

// after: check headroom before both clones
if !fd_headroom(4) {
    eprintln!("monitor: fd headroom too low for channel dups; raise ulimit -n");
}
let out_stream = flow_daemon::descr_of_out_channel(&server_handle.channels.1)
    .try_clone()
    .map_err(|e| { flow_daemon::close_noerr(&mut server_handle); e })?;

Defensive patterns

Strategy: validation

Validate before calling

let open = std::fs::read_dir("/proc/self/fd").map(|d| d.count()).unwrap_or(0);
if open + 8 >= fd_limit() {
    eprintln!("not enough fds to dup monitor channels; raising limit required");
    std::process::exit(1);
}

Prevention

Budget at least a handful of spare fds at startup, not zero
One fd leak is enough to land exactly between the two clones — audit leaks periodically
Raise nofile limits in the service manager, not just the shell

When it happens

Trigger: The same startup sequence as error 194 but failing one statement later: fd budget runs out exactly at the second clone, or the daemon closed only its write end before dying. Firing here means the first dup already consumed the last available fd.

Common situations: Monitors at exactly the RLIMIT_NOFILE boundary; fd leaks that grow by one between the two clones; containers with hard nofile caps.

Related errors

AI-assisted analysis of facebook/flow@f88ac94bcf (2026-08-20). Data as JSON: /api/errors/89addd9ea8fdde05. Report an issue: GitHub.