ZooKeeper response XID %d does not match request XID %d

Error message

ZooKeeper response XID %d does not match request XID %d

What it means

ZooKeeper multiplexes concurrent requests over one connection using an incrementing transaction id (XID); request() verifies the XID in the response equals the XID sent for the current request. This error means the reply frame's XID does not match — the response stream has desynchronized from the request stream (a response was lost, duplicated, or out of order, or a server-pushed event/watch payload was misread as a response).

Source

Thrown at agents/drivers/argo-go/zookeeper_protocol.go:323

	request.int32(client.xid)
	request.int32(opcode)
	if encodeBody != nil {
		encodeBody(request)
	}
	if err := client.writeFrame(request.data()); err != nil {
		return nil, err
	}
	response, err := client.readFrame()
	if err != nil {
		return nil, err
	}
	decoder := newZooKeeperDecoder(response)
	xid, err := decoder.int32()
	if err != nil {
		return nil, err
	}
	if xid != client.xid {
		return nil, fmt.Errorf("ZooKeeper response XID %d does not match request XID %d", xid, client.xid)
	}
	if _, err := decoder.int64(); err != nil {
		return nil, err
	}
	code, err := decoder.int32()
	if err != nil {
		return nil, err
	}
	if err := zooKeeperError(code); err != nil {
		return nil, err
	}
	return decoder.remaining(), nil
}

func (client *protocolZooKeeperClient) writeFrame(payload []byte) error {
	if len(payload) > zooKeeperMaxFrameSize {
		return fmt.Errorf("ZooKeeper request frame is %d bytes, maximum is %d", len(payload), zooKeeperMaxFrameSize)
	}

View on GitHub (pinned to c0390bff16)

Solutions

Close and reopen the ZooKeeper connection: once the stream desyncs, every subsequent response will mismatch — reconnecting is the only reliable recovery.
Avoid ignoring earlier request errors: if a previous operation failed with a timeout, treat the connection as poisoned and redial rather than issuing new requests on it.
Ensure only one request is in flight per connection — the client serializes under client.mutex, so check for other users of the same socket (e.g. a wrapper or shared client) issuing concurrent frames.
Check for proxies/load balancers that multiplex or buffer ZooKeeper traffic; connect directly to the quorum members.
Inspect whether watch/event packets from the server could be interleaved on this connection in your deployment and filter them before request responses.

Example fix

// before: reusing a connection after a timed-out request
_, err := client.Get("/key") // times out
_, err = client.Get("/other") // response XID 42 does not match request XID 43
// after: redial on timeout before issuing another request
if err != nil && isTimeout(err) {
    client.Close()
    client, err = zookeeper.Connect(hosts, timeout)
}

Defensive patterns

Strategy: try-catch

Try / catch

_, err := client.Get(path)
if err != nil {
    if strings.Contains(err.Error(), "does not match request XID") {
        // stream is desynced; the connection is unusable — redial
        client.Close()
        newClient, dialErr := zookeeper.Connect(hosts, timeout)
        if dialErr != nil {
            return dialErr
        }
        client = newClient
        return retryOperation()
    }
    return err
}

Prevention

Never reuse a connection after a timed-out request — treat timeouts as connection-fatal and redial.
Share the client only through its own mutex-serialized API; don't issue frames on the underlying socket from elsewhere.
Connect directly to quorum members; avoid multiplexing proxies in front of ZooKeeper.
Set read deadlines larger than worst-case server latency so responses are consumed before the client gives up.

When it happens

Trigger: client.request() (used by authenticateSASL, AddAuth, Children, Get) receives a frame whose leading int32 differs from client.xid: a previous response was dropped after a timeout while the server still sent it later, a frame from a prior/closed session arrives, or the byte stream is misaligned so a non-XID field is parsed as the XID.

Common situations: A previous call timed out at the read deadline but the server eventually replied, leaving the stale response in the socket buffer that the next request then reads; connection reuse across a reconnect without discarding buffered data; proxy/load-balancer interleaving frames; or session expiry causing the server to send a non-request-matched packet.

Related errors

AI-assisted analysis of t8y2/dbx@c0390bff16 (2026-09-05). Data as JSON: /api/errors/c0cfe4c834112866. Report an issue: GitHub.