xai-org/x-algorithm · error · VfChannelError::Connect

timed out after {}s

Error message

timed out after {}s

What it means

Returned in build_vf_channel when building the xDS channel for the visibility-filtering (VF) service exceeds VF_XDS_BUILD_TIMEOUT (the message states the timeout in seconds). The future build_xds_channel(params, &listener) was wrapped in a timeout and the deadline elapsed before the channel was ready.

Source

Thrown at visibility-filtering-client/discovery.rs:139

    Xds(LoadBalancedChannel),
}

pub(crate) async fn build_vf_channel(
    params: &VfChannelParams<'_>,
) -> Result<VfChannel, VfChannelError> {
    let name = params.name;
    match params.discovery {
        VfDiscovery::Wily => Ok(VfChannel::Wily(build_vf_wily_channel(params).await?)),
        VfDiscovery::Xds => {
            let listener = vf_xds_listener_from_env();
            let result = match tokio::time::timeout(
                VF_XDS_BUILD_TIMEOUT,
                build_xds_channel(params, &listener),
            )
            .await
            {
                Ok(result) => result,
                Err(_) => Err(VfChannelError::Connect(anyhow::anyhow!(
                    "timed out after {}s",
                    VF_XDS_BUILD_TIMEOUT.as_secs()
                ))),
            };
            match result {
                Ok(channel) => {
                    incr_build_metric(name, "xds", "built");
                    Ok(VfChannel::Xds(channel))
                }
                Err(e) => {
                    warn!(
                        channel = name,
                        listener = %listener,
                        error = %e,
                        "VF xDS channel build failed"
                    );
                    incr_build_metric(name, "xds", "failed");
                    Err(e)

View on GitHub (pinned to 24c60942c5)

Solutions

1. Check connectivity to the xDS server (host:port reachability, TLS certs valid)
2. Increase VF_XDS_BUILD_TIMEOUT if the environment legitimately needs longer (large configs, high latency)
3. Retry the build with backoff — transient xDS/control-plane slowness is common at startup
4. Verify xDS auth material (certs/tokens) referenced by params is present and unexpired

Example fix

// before
const VF_XDS_BUILD_TIMEOUT: Duration = Duration::from_secs(5);

// after
const VF_XDS_BUILD_TIMEOUT: Duration = Duration::from_secs(
    std::env::var("VF_XDS_BUILD_TIMEOUT_SECS").ok().and_then(|v| v.parse().ok()).unwrap_or(15),
);

// preflight: TCP-connect to xDS host:port before building the channel

for attempt in 0..3 { match build_vf_channel(params).await { Ok(c) => break Ok(c), Err(e) if attempt < 2 => sleep_backoff(attempt), Err(e) => break Err(e) } }

When it happens

Trigger: Calling build_vf_channel when xDS discovery is slow: unreachable xDS server, slow TLS handshake, large listener/config download, DNS resolution delays, or an overloaded control plane — anything making channel construction outlast VF_XDS_BUILD_TIMEOUT.

Common situations: xDS server down or misconfigured in the environment; network policies blocking gRPC/xDS ports in containers; cold start downloading big configs; control plane degradation during incidents.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• MH scan failed: {e}
• Rust VF filter_tweets error: {status}
• regex match being interrupted. Tmeout: %s.
• enable_stale_post is unsupported on grpc_recsys, which does
• PhoenixGrpcDataset requires server_address: a gRPC address p

AI-assisted analysis of xai-org/x-algorithm@24c60942c5 (2026-08-28). Data as JSON: /api/errors/88c6d98e79509881. Report an issue: GitHub.