vllm-project/vllm · error · ValueError

request {request.request_id!r}: request_id has no embedded p

Error message

request {request.request_id!r}: request_id has no embedded peer zmq_address and kv_transfer_params is missing remote_host / remote_notify_port (got remote_host={remote_host!r}, remote_notify_port={remote_notify_port!r})

What it means

Worker-side peer resolution during block notification: the request_id embeds no peer zmq_address and the sidecar fallback in kv_transfer_params yields a falsy remote_host or a remote_notify_port that converts to 0 (missing, None, or non-numeric). The error echoes both offending values so the bad field is identifiable.

Source

Thrown at vllm/distributed/kv_transfer/kv_connector/v1/moriio/moriio_connector.py:755

                    peer_zmq = get_peer_zmq_from_request_id(
                        request.request_id, is_producer=False
                    )
                    if peer_zmq is not None:
                        remote_host, _, remote_notify_port = parse_moriio_zmq_address(
                            peer_zmq
                        )
                    else:
                        # Sidecar fallback: use explicit params fields.
                        params = request.kv_transfer_params or {}
                        remote_host = params.get("remote_host") or ""
                        try:
                            remote_notify_port = int(
                                params.get("remote_notify_port") or 0
                            )
                        except (TypeError, ValueError):
                            remote_notify_port = 0
                        if not remote_host or not remote_notify_port:
                            raise ValueError(
                                f"request {request.request_id!r}: "
                                f"request_id has no embedded peer "
                                f"zmq_address and kv_transfer_params is "
                                f"missing remote_host / remote_notify_port "
                                f"(got remote_host={remote_host!r}, "
                                f"remote_notify_port={remote_notify_port!r})"
                            )

                    # num_external_tokens == 0: nothing to push, so don't tell
                    # the producer to write into these blocks.
                    block_notify_list = (
                        blocks.get_block_ids()[0] if num_external_tokens > 0 else []
                    )

                    # Wide-EP multi-pod: a pod binds notify sockets only for
                    # its LOCAL ranks, so the port offset must use the per-pod
                    # local rank (% dp_local), not the global rank. Single-pod
                    # is bit-identical (modulus is a no-op).

View on GitHub (pinned to c794754062)

Solutions

Populate both remote_host and remote_notify_port in kv_transfer_params for every MoRI-IO request
Read the echoed values in the message to see which field is '' or 0 and fix that side of the sidecar config
Restore/verify router-side zmq_address embedding in the request_id
Reject requests at admission when neither routing channel is complete

Example fix

// before
params = {"remote_host": "10.0.0.5", "remote_notify_port": None}

// after
params = {"remote_host": "10.0.0.5", "remote_notify_port": 5556}

Defensive patterns

Strategy: validation

Validate before calling

def can_route_request(request_id: str, params: dict | None) -> bool:
    if get_peer_zmq_from_request_id(request_id, is_producer=True) is not None:
        return True
    p = params or {}
    host = p.get("remote_host")
    port = p.get("remote_notify_port")
    return bool(host) and str(port or "").strip().isdigit() and int(port) > 0

Type guard

def has_worker_sidecar_routing(params) -> bool:
    return (
        isinstance(params, dict)
        and isinstance(params.get("remote_host"), str)
        and params["remote_host"].strip() != ""
        and str(params.get("remote_notify_port") or "").isdigit()
    )

Try / catch

try:
    notify_peer(...)
except ValueError as e:
    if "remote_notify_port" in str(e):
        drop_or_requeue_request(request_id, reason=str(e))  # routing incomplete, do not crash the worker
    else:
        raise

Prevention

Make the sidecar inject remote_host and remote_notify_port atomically with request submission
Prefer router-embedded zmq_address routing so the fallback rarely triggers
Alert on fallback-path usage: it usually signals router misconfiguration

When it happens

Trigger: MoRIIOConnectorWorker processing a request whose router did not embed routing, where params.get('remote_host') is empty or int(params.get('remote_notify_port') or 0) evaluates to 0 (key absent, None, '', or non-numeric string caught by the TypeError/ValueError handler).

Common situations: Sidecar injects only handshake-port fields and forgets the notify port; env-var-driven config where remote_notify_port is unset; router upgrade changed request_id embedding so requests fall into the fallback path unprepared.

Related errors

AI-assisted analysis of vllm-project/vllm@c794754062 (2026-08-14). Data as JSON: /api/errors/b3af754bffb3762d. Report an issue: GitHub.