What causes "timeout waiting for hybrid vsocket handshake of %s:%d" in containerd/containerd?

Increase the timeout parameter passed to AnonDialer/dialHybridVsock to cover guest startup time.

timeout waiting for hybrid vsocket handshake of %s:%d

Error message

timeout waiting for hybrid vsocket handshake of %s:%d

What it means

hybridVsockDialer retries the CONNECT handshake up to a deadline (10 retry intervals within the timeout) and also races the response reader against a timeout channel. If the deadline elapses before an "OK" response is read, the connection is closed and this timeout error is returned. It means the host bridge never completed the handshake in time.

Source

Thrown at pkg/shim/util_unix.go:240

				errChan <- fmt.Errorf("hybrid vsock handshake response error: %s", response)
			}
		}()
		select {
		case err = <-errChan:
			if err != nil {
				conn.Close()
				// When it is EOF, maybe the server side is not ready.
				if err == io.EOF {
					log.G(context.Background()).Warnf("Read hybrid vsock got EOF, server may not ready")
					time.Sleep(retryInterval)
					continue
				}
				return nil, err
			}
			return conn, nil
		case <-timeoutCh:
			conn.Close()
			return nil, fmt.Errorf("timeout waiting for hybrid vsocket handshake of %s:%d", addr, port)
		}
	}

}

func dialVsock(address string) (net.Conn, error) {
	contextIDString, portString, ok := strings.Cut(address, ":")
	if !ok {
		return nil, fmt.Errorf("invalid vsock address %s", address)
	}
	contextID, err := strconv.ParseUint(contextIDString, 10, 0)
	if err != nil {
		return nil, fmt.Errorf("failed to parse vsock context id %s, %v", contextIDString, err)
	}
	if contextID > math.MaxUint32 {
		return nil, fmt.Errorf("vsock context id %d is invalid", contextID)
	}
	port, err := strconv.ParseUint(portString, 10, 0)

View on GitHub (pinned to 4246446a2b)

Solutions

Increase the timeout parameter passed to AnonDialer/dialHybridVsock to cover guest startup time.
Verify the guest-side service on the requested port is running and accepting CONNECT requests.
Add caller-side retry with backoff around the dial for transient un readiness.
Check host proxy logs for stalled connections; restart the proxy if handshakes hang.

Example fix

// before
conn, err := shim.AnonDialer(addr, 100*time.Millisecond)
// after
conn, err := shim.AnonDialer(addr, 10*time.Second)

Defensive patterns

Strategy: retry

Validate before calling

// Probe readiness with a generous timeout before the real dial:
if !shim.CanConnect(hvAddr) { // internally uses a 100ms probe
	time.Sleep(2 * time.Second) // guest not ready yet
}

Type guard

func isHvsockHandshakeTimeout(err error) bool {
	return err != nil && strings.Contains(err.Error(), "timeout waiting for hybrid vsocket handshake")
}

Try / catch

var conn net.Conn
var err error
for i := 0; i < 5; i++ {
	conn, err = shim.AnonDialer(hvAddr, 10*time.Second)
	if err == nil || !isHvsockHandshakeTimeout(err) {
		break
	}
	time.Sleep(time.Duration(1<<i) * time.Second)
}

Prevention

Pass timeouts long enough to cover guest boot (seconds, not milliseconds).
Retry with exponential backoff around hvsock dials during VM startup.
Monitor the host hvsock proxy for hung connections and restart if stalled.

When it happens

Trigger: AnonDialer/AnonReconnectDialer with an 'hvsock://' address where the proxy accepts the TCP/unix connection but never sends "OK" within the provided timeout — guest not ready, EOF-retry loop exhausting the deadline, or a hung proxy.

Common situations: Dialing a hybrid vsock shim immediately after VM start before the guest service binds; passing a too-short timeout (e.g. 100ms); slow guest boot under load.

Understand the failure class

SSL/TLS and certificate errors — how TLS handshakes and certificate validation fail.
Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

AI-assisted analysis of containerd/containerd@4246446a2b (2026-09-02). Data as JSON: /api/errors/7dc3267ef1bd1e0c. Report an issue: GitHub.