{"record":{"id":"4ff951fa9624fcf8","repo":"vllm-project/vllm","slug":"unexpected-frame-received-frame","errorCode":null,"errorMessage":"Unexpected frame! {received_frame = }","messagePattern":"Unexpected frame! (.+?)","errorType":"exception","errorClass":"HandshakeError","httpStatus":null,"severity":"error","filePath":"vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_connector.py","lineNumber":1547,"sourceCode":"        # a hack to keep us moving. We will switch when moving to etcd\n        # or where we have a single ZMQ socket in the scheduler.\n\n        dial_tp_rank = (\n            self._remote_tp_rank(remote_tp_size)\n            if remote_tp_rank is None\n            else int(remote_tp_rank)\n        )\n        port_offset = get_port_offset(remote_dp_rank, dial_tp_rank, remote_tp_size)\n        path = make_zmq_path(\"tcp\", host, port + port_offset)\n        logger.debug(\"handshake Querying metadata on path: %s\", path)\n\n        # Send query for the request.\n        with zmq_ctx(zmq.DEALER, path) as sock:\n            logger.debug(\"prepare send msg INSTAZNCE: %s\", path)\n            sock.send(MoRIIOConstants.GET_META_MSG)\n            received_frame = sock.recv_multipart()\n            if len(received_frame) != 2 or received_frame[0] != b\"\":\n                raise HandshakeError(f\"Unexpected frame! {received_frame = }\")\n\n            metadata_bytes = received_frame[1]\n            decoder = msgspec.msgpack.Decoder(MoRIIOAgentMetadata)\n            metadata = decoder.decode(metadata_bytes)\n            got_metadata_time = time.perf_counter()\n            logger.info(\n                \"MoRIIO handshake: get metadata took: %s\",\n                got_metadata_time - start_time,\n            )\n\n            self.moriio_wrapper.remote_engine_ip = host\n            remote_agent_name = self.moriio_wrapper.register_remote_engine(\n                metadata.agent_metadata\n            )\n\n            logger.debug(\n                \"MoRIIO handshake: registered\"\n                \"remote agent %s for engine ID %s, path = %s\",","sourceCodeStart":1529,"sourceCodeEnd":1565,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_connector.py#L1529-L1565","documentation":"During the DEALER-side handshake, after sending GET_META_MSG the client expects exactly a two-part frame [b'', metadata_payload]. Receiving a different frame count or a non-empty first (delimiter) part raises HandshakeError, meaning the reply does not come from a compatible MoRI-IO ROUTER.","triggerScenarios":"Connecting the DEALER to a port served by something else (another ZMQ service, a REP/PUB socket), a peer with mismatched protocol version, or a peer that errors out mid-reply so framing is off.","commonSituations":"Wrong host/port or wrong port offset in the peer address; port collisions in multi-pod deployments; version skew between vLLM builds changing message framing.","solutions":["Verify the DEALER connects to the exact port the peer ROUTER bound (base port + dp/tp offset)","Ensure both sides run the same vLLM/MoRI-IO version","Check the echoed received_frame in the message: single-part frames usually mean a non-ROUTER peer","Eliminate port-range overlaps between instances/ranks"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"# Validate the computed peer endpoint before connecting.\ndef expected_peer_endpoint(host: str, base_port: int, dp_rank: int, tp_rank: int, tp_size: int) -> str:\n    from vllm.distributed.kv_transfer.kv_connector.v1.moriio.moriio_common import make_zmq_path, get_port_offset\n    return make_zmq_path(\"tcp\", host, base_port + get_port_offset(dp_rank, tp_rank, tp_size))","typeGuard":null,"tryCatchPattern":"try:\n    metadata = query_remote_metadata(path)\nexcept HandshakeError as e:\n    if \"Unexpected frame\" in str(e):\n        verify_peer_version_and_port_allocation(); reconnect_with_backoff()\n    else:\n        raise","preventionTips":["Compute and log the exact peer path (host + port + offset) on both sides and diff them","Keep vLLM/MoRI-IO versions aligned across the cluster","Allocate disjoint port ranges per DP/TP rank to avoid hitting a foreign socket"],"tags":["zmq","handshake","protocol-mismatch","kv-transfer"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}