{"record":{"id":"55c8fb7abc9337b7","repo":"vllm-project/vllm","slug":"handshake-failed-unexpected-msg-type","errorCode":null,"errorMessage":"handshake failed, unexpected msg type","messagePattern":"handshake failed, unexpected msg type","errorType":"exception","errorClass":"HandshakeError","httpStatus":null,"severity":"error","filePath":"vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_connector.py","lineNumber":1492,"sourceCode":"            \"Size of encoded MoRIIOAgentMetadata: %s bytes\", str(size_in_bytes)\n        )\n\n        # Listen for new requests for metadata.\n        host = \"*\"\n\n        path = make_zmq_path(\"tcp\", host, base_port)\n        logger.debug(\"mori handshake starting listening on path: %s\", path)\n\n        with zmq_ctx(zmq.ROUTER, path) as sock:\n            ready_event.set()\n            while True:\n                identity, msg = sock.recv_multipart()\n                if (\n                    msg != MoRIIOConstants.GET_META_MSG\n                    and msg != MoRIIOConstants.POP_DONE_RECV\n                ):\n                    logger.error(\"Connection listener got unexpected message\")\n                    raise HandshakeError(\"handshake failed, unexpected msg type\")\n                elif msg == MoRIIOConstants.GET_META_MSG:\n                    sock.send_multipart(\n                        (identity, b\"\", encoded_data)\n                    )  # send local mori io engine meta data\n                    logger.debug(\"MoRIIO handshake listener sent metadata\")\n                    # now we send tensor meta data for each block\n                    buf = msgpack.dumps(layer_name_to_local_kv_cache_metadata)\n                    sock.send_multipart((identity, b\"\", buf))\n                elif msg == MoRIIOConstants.POP_DONE_RECV:\n                    _, req_id = sock.recv_multipart()\n                    logger.debug(\n                        \"MoRIIO handshake listener received done recv for req\",\n                        req_id.decode(),\n                    )\n\n    def _moriio_handshake(\n        self,\n        host: str,","sourceCodeStart":1474,"sourceCodeEnd":1510,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_connector.py#L1474-L1510","documentation":"The handshake ROUTER listener only accepts two message types from peers: GET_META_MSG (metadata query) and POP_DONE_RECV (transfer-done notification). Any other first frame raises HandshakeError, aborting the listener thread and thus the connector's handshake capability.","triggerScenarios":"A client connecting to the handshake port and sending an unexpected/undecodable frame: vLLM version skew where constant values differ, a non-MoRI-IO ZMQ client hitting the port, or stream corruption delivering garbage bytes.","commonSituations":"Mixed vLLM versions on prefill and decode with divergent MoRI-IO protocol constants; port reuse where another service or another DP/TP rank's socket connects to the wrong listener; manual ZMQ probing against the handshake port.","solutions":["Pin both P and D instances to the same vLLM version so protocol constants match","Verify the connecting peer is a MoRI-IO instance and computes the same port offset (dp_rank/tp_rank based)","Reserve disjoint port ranges per rank/instance to avoid cross-connections","Inspect the logged 'Connection listener got unexpected message' frame bytes to identify which client sent it"],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"# Nothing to validate locally; verify peer identity/version before connecting.\ndef peer_version_matches(remote_version: str, local_version: str) -> bool:\n    return remote_version == local_version","typeGuard":null,"tryCatchPattern":"try:\n    run_handshake_listener(...)\nexcept HandshakeError as e:\n    if \"unexpected msg type\" in str(e):\n        log_offending_frame_and_peer_identity()\n        restart_listener_after_peer_version_check()\n    else:\n        raise","preventionTips":["Run identical vLLM versions on all P/D instances so protocol constants match","Reserve exclusive port ranges per rank/instance to prevent foreign clients","Do not point arbitrary ZMQ tools at the handshake port"],"tags":["zmq","handshake","protocol-mismatch","kv-transfer"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}