vllm-project/vllm · error · ValueError

request_id {request_id!r} does not embed a peer zmq_address

Error message

request_id {request_id!r} does not embed a peer zmq_address and kv_transfer_params['remote_host'] is empty; cannot route MoRI-IO transfer

What it means

Raised while resolving the peer for a MoRI-IO transfer: the request_id does not embed a peer zmq_address (get_peer_zmq_from_request_id returned None) and the explicit fallback kv_transfer_params['remote_host'] is present but empty/falsy. Without a remote host the connector cannot route the transfer.

Source

Thrown at vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_common.py:531

        Routing keys consumed elsewhere (NOT here): ``remote_dp_rank`` /
        ``remote_dp_rank_override`` gate the decode->prefill notify target in
        MoRIIOConnectorScheduler; they are router-authoritative and never
        self-derived (see that class's request routing contract).
        """
        transfer_id = kv_transfer_params["transfer_id"]

        # Try request_id embedded address first, fallback to explicit params.
        peer_zmq = get_peer_zmq_from_request_id(request_id, is_producer=write_mode)
        if peer_zmq is not None:
            remote_host, remote_handshake_port, remote_notify_port = (
                parse_moriio_zmq_address(peer_zmq)
            )
        else:
            try:
                remote_host = kv_transfer_params["remote_host"]
                if not remote_host:
                    raise ValueError(
                        f"request_id {request_id!r} does not embed a peer "
                        f"zmq_address and kv_transfer_params['remote_host'] is "
                        f"empty; cannot route MoRI-IO transfer"
                    )
                remote_handshake_port = int(kv_transfer_params["remote_handshake_port"])
                remote_notify_port = int(kv_transfer_params["remote_notify_port"])
            except (KeyError, TypeError, ValueError) as e:
                raise ValueError(
                    f"request_id {request_id!r} does not embed a peer "
                    f"zmq_address and kv_transfer_params is missing one or "
                    f"more sidecar-fallback keys (need remote_host, "
                    f"remote_handshake_port, remote_notify_port): {e}"
                ) from e

        # Multi-pod: use multi_pod_hosts list or fallback to single host.
        _pod_hosts = kv_transfer_params.get("remote_hosts") or [remote_host]
        if not isinstance(_pod_hosts, list):
            _pod_hosts = [_pod_hosts]

View on GitHub (pinned to c794754062)

Solutions

1. Set a non-empty remote_host in kv_transfer_params (and valid remote_handshake_port / remote_notify_port)
2. Or restore router-side address embedding in the request_id so the fallback path is not used
3. Check the sidecar/injector code for why host resolves to an empty string (missing env var, wrong config key)
4. Fail fast at request admission when neither routing source is available

Example fix

// before
kv_transfer_params = {"remote_host": "", "remote_handshake_port": 5555, "remote_notify_port": 5556}

// after
kv_transfer_params = {"remote_host": "10.0.0.5", "remote_handshake_port": 5555, "remote_notify_port": 5556}

def has_complete_remote_routing(params: dict) -> bool:
    if get_peer_zmq_from_request_id(request_id, is_producer=write_mode) is not None:
        return True
    return bool(params.get("remote_host")) and bool(params.get("remote_handshake_port")) and bool(params.get("remote_notify_port"))

def is_non_empty_host(v) -> bool:
    return isinstance(v, str) and v.strip() != ""

try:
    resolve_peer(...)
except ValueError as e:
    if "remote_host" in str(e) and "empty" in str(e):
        # reject/repair request routing instead of crashing the worker
        mark_request_unroutable(request_id)
    else:
        raise

When it happens

Trigger: Prefill/decode disagg where the router does not embed the peer address in the request_id, and the sidecar or client passes kv_transfer_params with remote_host='' or None while still providing the port keys.

Common situations: Sidecar populates ports from one config source but the host from another that is unset; empty-string host after a template/env var substitution failure; switching from router-embedded routing to explicit params without fully populating them.

Related errors

• request {request.request_id!r}: request_id has no embedded p
• request_id {request_id!r} does not embed a peer zmq_address
• --use-replayssm is incompatible with KV connectors (P/D disa
• kv_transfer_config must be set to create a connector
• Connector {connector_cls.__name__} does not support HMA but

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/50a503b4dab145e3. Report an issue: GitHub.