vllm-project/vllm · error · ValueError

request_id {request_id!r} does not embed a peer zmq_address

Error message

request_id {request_id!r} does not embed a peer zmq_address and kv_transfer_params is missing one or more sidecar-fallback keys (need remote_host, remote_handshake_port, remote_notify_port): {e}

What it means

Raised in the same peer-resolution fallback path: the request_id carries no embedded zmq_address and kv_transfer_params is missing one or more of remote_host, remote_handshake_port, remote_notify_port (KeyError), has None values (TypeError), or non-numeric ports (ValueError). The chained exception {e} names the failing key or conversion.

Source

Thrown at vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_common.py:539

        # Try request_id embedded address first, fallback to explicit params.
        peer_zmq = get_peer_zmq_from_request_id(request_id, is_producer=write_mode)
        if peer_zmq is not None:
            remote_host, remote_handshake_port, remote_notify_port = (
                parse_moriio_zmq_address(peer_zmq)
            )
        else:
            try:
                remote_host = kv_transfer_params["remote_host"]
                if not remote_host:
                    raise ValueError(
                        f"request_id {request_id!r} does not embed a peer "
                        f"zmq_address and kv_transfer_params['remote_host'] is "
                        f"empty; cannot route MoRI-IO transfer"
                    )
                remote_handshake_port = int(kv_transfer_params["remote_handshake_port"])
                remote_notify_port = int(kv_transfer_params["remote_notify_port"])
            except (KeyError, TypeError, ValueError) as e:
                raise ValueError(
                    f"request_id {request_id!r} does not embed a peer "
                    f"zmq_address and kv_transfer_params is missing one or "
                    f"more sidecar-fallback keys (need remote_host, "
                    f"remote_handshake_port, remote_notify_port): {e}"
                ) from e

        # Multi-pod: use multi_pod_hosts list or fallback to single host.
        _pod_hosts = kv_transfer_params.get("remote_hosts") or [remote_host]
        if not isinstance(_pod_hosts, list):
            _pod_hosts = [_pod_hosts]
        _pod_hosts = [str(h) for h in _pod_hosts]
        _remote_dp_size_local = int(
            kv_transfer_params.get(
                "remote_dp_size_local",
                kv_transfer_params.get("remote_dp_size", 1),
            )
        )

View on GitHub (pinned to c794754062)

Solutions

1. Read the chained {e} to identify the exact missing key or bad value
2. Supply all three keys in kv_transfer_params: remote_host (non-empty str), remote_handshake_port and remote_notify_port as ints or numeric strings
3. Or enable router-side request_id address embedding so the fallback is unnecessary
4. Validate the params dict shape at request build time, not at transfer time

Example fix

// before
params = {"remote_host": "10.0.0.5"}  # ports missing -> KeyError

// after
params = {"remote_host": "10.0.0.5", "remote_handshake_port": 5555, "remote_notify_port": 5556}

REQUIRED_REMOTE_KEYS = ("remote_host", "remote_handshake_port", "remote_notify_port")

def validate_sidecar_fallback(params: dict) -> None:
    for k in REQUIRED_REMOTE_KEYS:
        if not params.get(k):
            raise ValueError(f"kv_transfer_params missing {k}; supply all of {REQUIRED_REMOTE_KEYS}")
    int(params["remote_handshake_port"]); int(params["remote_notify_port"])

def has_sidecar_fallback_keys(params) -> bool:
    return (
        isinstance(params, dict)
        and isinstance(params.get("remote_host"), str)
        and params["remote_host"].strip() != ""
        and str(params.get("remote_handshake_port", "")).isdigit()
        and str(params.get("remote_notify_port", "")).isdigit()
    )

try:
    remote_host, hs, nf = resolve_peer_routing(request_id, kv_transfer_params)
except ValueError as e:
    # read chained exception to find the exact missing key, then reject request
    reject_request(request_id, reason=str(e))

When it happens

Trigger: Request admission without router-embedded routing plus kv_transfer_params that omits any of the three sidecar keys, sets them to None, or supplies ports as non-integer strings like '55five' or 'tcp://host:5555'.

Common situations: Sidecar version drift where new required keys were added; populating params from a JSON that uses different key names; passing the whole zmq address string as remote_host so the int() port conversion is never reached cleanly.

Understand the failure class

• SSL/TLS and certificate errors — how TLS handshakes and certificate validation fail.

Related errors

• kv_connector_module_path cannot be an empty string.
• Invalid MoRIIO backend {backend!r} in kv_connector_extra_con
• Malformed zmq_address {zmq_address!r}: expected 'host:IP,han
• request_id {request_id!r} does not embed a peer zmq_address
• TP sizes must be positive

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/c856e045e8487009. Report an issue: GitHub.