VictoriaMetrics/VictoriaMetrics · error

cannot reset deadline: %w

Error message

cannot reset deadline: %w

What it means

This error wraps a failure from `c.SetDeadline(time.Time{})` at the very end of genericServer, which clears the handshake timeout so the connection can be used for normal RPC traffic. The library throws it when the underlying net.Conn refuses or fails the deadline-reset call after a successful handshake exchange. In practice this only happens on already-broken connections (closed, unsupported conn types, or internal deadline bookkeeping failures), since clearing a deadline rarely fails on a healthy TCP conn.

Source

Thrown at lib/handshake/handshake.go:229

	if err := writeMessage(c, successResponse); err != nil {
		return nil, fmt.Errorf("cannot write success response on isCompressed: %w", err)
	}
	isRemoteCompressed, err := readIsCompressed(c)
	if err != nil {
		return nil, fmt.Errorf("cannot read isCompressed flag: %w", err)
	}
	if err := writeMessage(c, successResponse); err != nil {
		return nil, fmt.Errorf("cannot write success response on isCompressed: %w", err)
	}
	if err := writeIsCompressed(c, compressionLevel > 0); err != nil {
		return nil, fmt.Errorf("cannot write isCompressed flag: %w", err)
	}
	if err := readMessage(c, successResponse); err != nil {
		return nil, fmt.Errorf("cannot read success response on isCompressed: %w", err)
	}

	if err := c.SetDeadline(time.Time{}); err != nil {
		return nil, fmt.Errorf("cannot reset deadline: %w", err)
	}

	bc := newBufferedConn(c, compressionLevel, isRemoteCompressed)
	return bc, nil
}

func genericClient(c net.Conn, msg string, compressionLevel int) (*BufferedConn, error) {
	if err := c.SetDeadline(time.Now().Add(*rpcHandshakeTimeout)); err != nil {
		return nil, fmt.Errorf("cannot set deadline: %w", err)
	}

	if err := writeMessage(c, msg); err != nil {
		return nil, fmt.Errorf("cannot write hello: %w", err)
	}
	if err := readMessage(c, successResponse); err != nil {
		return nil, fmt.Errorf("cannot read success response after sending hello: %w", err)
	}
	if err := writeIsCompressed(c, compressionLevel > 0); err != nil {

View on GitHub (pinned to 5079fb58f1)

Solutions

1. Check client-side logs for why the peer closed the connection during handshake; fix the client that hangs up early.
2. Verify any custom net.Conn implementation correctly supports SetDeadline (including time.Time{} to clear).
3. Rule out intermediaries (load balancers, k8s probes) with short idle/close timeouts that race with handshake completion.
4. Retry the connection; this error is transient when caused by a disconnect race.

// client side: ensure the conn stays open through the whole handshake
if err := conn.SetDeadline(time.Now().Add(timeout)); err != nil {
    return fmt.Errorf("conn unusable before handshake: %w", err)
}

bc, err := handshake.VMSelectServer(conn, compressionLevel)
if err != nil {
    if strings.Contains(err.Error(), "cannot reset deadline") {
        // transient peer disconnect; re-accept/retry
        return retryAccept()
    }
    return err
}

When it happens

Trigger: Any call into genericServer — VMInsertServer, VMInsertServerWithLegacyHello, or VMSelectServer — where the server-side handshake completed (hello read, success responses written, isCompressed flags exchanged) but the final SetDeadline(zero) call returns an error, typically because the peer closed the connection mid-handshake or the conn is in an error state.

Common situations: The client disconnected immediately after the last handshake byte (race between handshake completion and client teardown); a proxy/LB killed the connection between the last read and the deadline reset; a custom net.Conn wrapper that mishandles SetDeadline is used in tests or custom transports.

Related errors

• cannot write success response on isCompressed: %w
• cannot write hello: %w
• cannot read success response after sending hello: %w
• cannot write %q to server: %w
• cannot send empty error message: %w

AI-assisted analysis of VictoriaMetrics/VictoriaMetrics@5079fb58f1 (2026-09-03). Data as JSON: /api/errors/b49cb388de1e64ac. Report an issue: GitHub.