tracel-ai/burn · critical

Server disconnected during initialization

Error message

Server disconnected during initialization

What it means

During session init the client sends `RemoteMessage::Init` on the request stream and waits for one message on the response stream. `response.recv()` returns `Ok(None)` when the stream was closed by the peer without delivering a message; this expect turns that into a panic. It means the server accepted the connection but terminated the response stream before answering the handshake.

Source

Thrown at crates/burn-remote/src/client/service.rs:179

        request: &mut SubmitChannel,
        response: &mut ResponseChannel,
        endpoint: &RemoteEndpoint,
        session_id: SessionId,
        device_index: u32,
    ) -> (DeviceSettings, u32) {
        let init_bytes: bytes::Bytes = rmp_serde::to_vec(&vec![RemoteMessage::Init(
            SessionInit::new(session_id, device_index, endpoint.authorization().to_vec()),
        )])
        .expect("Can serialize RemoteMessage::Init")
        .into();

        let result: Result<(DeviceSettings, u32), String> = async {
            request.send(init_bytes).await?;

            let msg = response
                .recv()
                .await?
                .expect("Server disconnected during initialization");
            let reply: TaskResponse =
                rmp_serde::from_slice(&msg).expect("Can deserialize init handshake payload");

            match reply.content {
                TaskResponseContent::Init(SessionInfo {
                    version,
                    settings,
                    device_count,
                    ..
                }) => {
                    if version != PROTOCOL_VERSION {
                        panic!(
                            "Server uses Burn Remote protocol version {version}, expected {PROTOCOL_VERSION}"
                        );
                    }
                    Ok((settings, device_count))
                }
                other => panic!("Expected Init response, got {other:?}"),

View on GitHub (pinned to d16f7ba2ed)

Solutions

1. Check the server is running and inspect its logs for a crash or rejection at handshake time.
2. Verify the server address/port passed to the RemoteEndpoint points at a burn-remote server of the matching version.
3. Confirm the authorization token in the endpoint matches the server's configured token.
4. Add network-level checks: keepalives/firewall rules/proxy timeouts so the stream is not closed during setup.
5. Retry connecting — transient server restarts produce exactly this one-shot failure.

Example fix

// before: fire-and-forget connect to a possibly stale address
let device = RemoteDevice::new(&"ws://stale-host:3000".parse().unwrap());
// after: ensure the server is up and address is correct before connecting
assert!(server_addr reachable); // e.g. TCP probe / health check
let device = RemoteDevice::new(&current_endpoint_from_config());

// Probe the server before creating a remote device:
use std::net::TcpStream;
fn server_reachable(addr: std::net::SocketAddr) -> bool {
    TcpStream::connect_timeout(&addr, std::time::Duration::from_secs(3)).is_ok()
}
// Also verify process liveness (systemd status / k8s probe) and that auth token matches server config.

// handshake_async surfaces the failure as a String error then panics; isolate init:
let result = std::panic::catch_unwind(|| RemoteService::init(...));
match result {
    Ok(svc) => svc,
    Err(_) => { log::warn!("server dropped init handshake; retrying"); backoff_retry(init, 3); }
}

When it happens

Trigger: The burn-remote server process crashes or exits between accepting the channels and replying to the Init message; the server closes the connection due to failed authorization or an immediate internal error; a proxy/firewall or idle timeout kills the connection right after it opens; connecting to a port where a non-burn server is listening and then closes the stream.

Common situations: Server deployed with a wrong authorization token and drops the session; server OOM-killed or restarted mid-handshake; connecting to the wrong host/port (e.g. a plain TCP echo or a different service); Kubernetes/load-balancer idle timeout shorter than connection setup; wasm client hitting a server behind a reverse proxy that buffers websockets.

Related errors

• Can deserialize init handshake payload
• Failed to open remote 'data' channel to {address}: {err:?}.
• Failed to receive message from websocket: {err:?}
• Failed to close WebSocket stream
• Failed to send download id

AI-assisted analysis of tracel-ai/burn@d16f7ba2ed (2026-09-05). Data as JSON: /api/errors/4df974a0eb9a8beb. Report an issue: GitHub.