vllm-project/vllm · error · HandshakeError

Unexpected frame! {received_frame = }

Error message

Unexpected frame! {received_frame = }

What it means

During the DEALER-side handshake, after sending GET_META_MSG the client expects exactly a two-part frame [b'', metadata_payload]. Receiving a different frame count or a non-empty first (delimiter) part raises HandshakeError, meaning the reply does not come from a compatible MoRI-IO ROUTER.

Source

Thrown at vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_connector.py:1547

        # a hack to keep us moving. We will switch when moving to etcd
        # or where we have a single ZMQ socket in the scheduler.

        dial_tp_rank = (
            self._remote_tp_rank(remote_tp_size)
            if remote_tp_rank is None
            else int(remote_tp_rank)
        )
        port_offset = get_port_offset(remote_dp_rank, dial_tp_rank, remote_tp_size)
        path = make_zmq_path("tcp", host, port + port_offset)
        logger.debug("handshake Querying metadata on path: %s", path)

        # Send query for the request.
        with zmq_ctx(zmq.DEALER, path) as sock:
            logger.debug("prepare send msg INSTAZNCE: %s", path)
            sock.send(MoRIIOConstants.GET_META_MSG)
            received_frame = sock.recv_multipart()
            if len(received_frame) != 2 or received_frame[0] != b"":
                raise HandshakeError(f"Unexpected frame! {received_frame = }")

            metadata_bytes = received_frame[1]
            decoder = msgspec.msgpack.Decoder(MoRIIOAgentMetadata)
            metadata = decoder.decode(metadata_bytes)
            got_metadata_time = time.perf_counter()
            logger.info(
                "MoRIIO handshake: get metadata took: %s",
                got_metadata_time - start_time,
            )

            self.moriio_wrapper.remote_engine_ip = host
            remote_agent_name = self.moriio_wrapper.register_remote_engine(
                metadata.agent_metadata
            )

            logger.debug(
                "MoRIIO handshake: registered"
                "remote agent %s for engine ID %s, path = %s",

View on GitHub (pinned to c794754062)

Solutions

1. Verify the DEALER connects to the exact port the peer ROUTER bound (base port + dp/tp offset)
2. Ensure both sides run the same vLLM/MoRI-IO version
3. Check the echoed received_frame in the message: single-part frames usually mean a non-ROUTER peer
4. Eliminate port-range overlaps between instances/ranks

# Validate the computed peer endpoint before connecting.
def expected_peer_endpoint(host: str, base_port: int, dp_rank: int, tp_rank: int, tp_size: int) -> str:
    from vllm.distributed.kv_transfer.kv_connector.v1.moriio.moriio_common import make_zmq_path, get_port_offset
    return make_zmq_path("tcp", host, base_port + get_port_offset(dp_rank, tp_rank, tp_size))

try:
    metadata = query_remote_metadata(path)
except HandshakeError as e:
    if "Unexpected frame" in str(e):
        verify_peer_version_and_port_allocation(); reconnect_with_backoff()
    else:
        raise

When it happens

Trigger: Connecting the DEALER to a port served by something else (another ZMQ service, a REP/PUB socket), a peer with mismatched protocol version, or a peer that errors out mid-reply so framing is off.

Common situations: Wrong host/port or wrong port offset in the peer address; port collisions in multi-pod deployments; version skew between vLLM builds changing message framing.

Related errors

• handshake failed, unexpected msg type
• unexpected frame! {received_frame = }
• tokenize endpoint unavailable: {0}
• {type(self).__name__} received pp_rank > 0 handshake metadat
• Malformed zmq_address {zmq_address!r}: expected 'host:IP,han

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/4ff951fa9624fcf8. Report an issue: GitHub.