{"record":{"id":"8293a49d1090d057","repo":"vllm-project/vllm","slug":"tokenize-endpoint-unavailable-0","errorCode":null,"errorMessage":"tokenize endpoint unavailable: {0}","messagePattern":"tokenize endpoint unavailable: (.+?)","errorType":"exception","errorClass":"BenchError","httpStatus":null,"severity":"error","filePath":"rust/src/bench/src/error.rs","lineNumber":21,"sourceCode":"\nuse thiserror::Error;\n\n#[derive(Error, Debug)]\npub enum BenchError {\n    #[error(\"HTTP request failed: {0}\")]\n    Http(#[from] reqwest::Error),\n\n    #[error(\"JSON error: {0}\")]\n    Json(#[from] serde_json::Error),\n\n    #[error(\"Tokenizer error: {0}\")]\n    Tokenizer(String),\n\n    /// The server's /tokenize//detokenize endpoint is not usable (4xx status:\n    /// not exposed, or rejected by a gateway such as LLM-d/EPP that returns\n    /// 400 instead of 404). Callers treat this as \"skip verification\", unlike\n    /// `Tokenizer` errors which are genuine failures.\n    #[error(\"tokenize endpoint unavailable: {0}\")]\n    TokenizeUnavailable(String),\n\n    #[error(\"Configuration error: {0}\")]\n    Config(String),\n\n    #[error(\"Endpoint not ready after {0}s: {1}\")]\n    EndpointTimeout(u64, String),\n\n    #[error(\"Backend error: {0}\")]\n    Backend(String),\n\n    #[error(\"IO error: {0}\")]\n    Io(#[from] std::io::Error),\n}\n\npub type Result<T> = std::result::Result<T, BenchError>;\n","sourceCodeStart":3,"sourceCodeEnd":38,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/rust/src/bench/src/error.rs#L3-L38","documentation":"Logged then raised as HandshakeError in the _async_wait ZMQ ROUTER listener thread (moriio_engine.py:719) when _handle_message throws while processing an incoming notify message from a remote node. _handle_message first tries msgpack.loads for structured messages (remote_blocks / write_done / release) and falls back to UTF-8 string completions; unhandled formats raise MoRIIOError, and the structured handlers themselves raise MoRIIOError on bad payloads (empty block_notify_list, invalid consumer_tp_size) or AssertionError on role violations (e.g. decode receiving a release message). The listener converts any of these to HandshakeError, which kills the daemon notify thread — after which the node silently stops receiving transfer notifications.","triggerScenarios":"A remote peer sends a message this node cannot parse or refuses: msgpack payload with an unknown 'type' (version skew between prefill and decode MoRIIO builds), a 'remote_blocks' message arriving at a decode-role node or a 'release'/'write_done' at the wrong role (assertion inside _handle_*_message), a structured message with missing 'transfer_id', or garbage bytes that are neither valid msgpack nor UTF-8. Any single bad message terminates the notify thread via the 'raise' after logging.","commonSituations":"Prefill and decode instances running different vLLM/MoRIIO versions where the notify protocol gained new message types; a peer from an older deployment still sending bare-string completions after the node upgraded to structured-only handling; role misconfiguration (e.g. both sides set as producers) so role asserts fire; a non-MoRIIO process or port scanner connecting to the notify port and sending arbitrary bytes; message corruption over the wire.","solutions":["Check the log line immediately preceding the HandshakeError — _handle_message logs 'Failed to decode msgpack message, will try as string' or 'Received non-UTF8 message' and the repr of the offending payload, which identifies the exact bad message.","Verify prefill and decode run the same vLLM/MoRIIO version so notify message schemas match; upgrade the older side.","Confirm role configuration is correct (one PRODUCER/prefill, one decode consumer) — role mismatches trip assertions inside the structured handlers.","Make sure nothing else (monitoring probe, scanner, stale process) can connect to the ZMQ notify port and inject non-protocol bytes.","Restart the affected engine process to restart the notify listener thread — it is a daemon thread and will not come back on its own after the raise."],"exampleFix":"// before (any bad message kills the notify thread)\nexcept Exception as e:\n    logger.error(\"Error processing message: %s\", e)\n    raise HandshakeError(f\"Error processing message: {e}\") from e\n\n// after (quarantine malformed messages, keep listener alive)\nexcept Exception as e:\n    logger.exception(\"Dropping malformed notify message from %s: %r\", identity, msg)\n    continue  # or count and abort after N consecutive failures","handlingStrategy":"validation","validationCode":"import msgpack\n\nKNOWN_MESSAGE_TYPES = {\"remote_blocks\", \"write_done\", \"release\"}\n\ndef is_valid_notify_payload(msg: bytes) -> bool:\n    try:\n        data = msgpack.loads(msg)\n    except Exception:\n        return True  # legacy bare-string completions are valid\n    if not isinstance(data, dict):\n        return False\n    mtype = data.get(\"type\", \"remote_blocks\" if \"req_id\" in data else None)\n    return mtype in KNOWN_MESSAGE_TYPES and \"transfer_id\" in data\n\n# e.g. gate before delegating to the engine's handler:\n# if not is_valid_notify_payload(msg): log_and_drop(msg)","typeGuard":"from typing import TypeGuard, Any\nimport msgpack\n\ndef is_known_structured_message(msg: bytes) -> TypeGuard[dict[str, Any]] | bool:\n    try:\n        data = msgpack.loads(msg)\n    except Exception:\n        return False\n    if not isinstance(data, dict):\n        return False\n    t = data.get(\"type\", \"remote_blocks\" if \"req_id\" in data else None)\n    return t in {\"remote_blocks\", \"write_done\", \"release\"}","tryCatchPattern":null,"preventionTips":["Keep prefill and decode instances on the same vLLM/MoRIIO version so the ZMQ notify protocol schema matches.","Restrict network access to the notify port (firewall/security group) so only known peers can connect; scanners produce unparseable bytes.","Verify role assignment (producer=prefill, consumer=decode) before starting; role asserts inside message handlers kill the listener thread.","Watch for the notify thread's death: after a HandshakeError the daemon thread exits and no further notifications arrive — alert on missing notify activity rather than waiting for timeouts downstream.","If you control a fork of the engine, consider dropping-and-counting malformed messages instead of re-raising, so one bad frame cannot silence the node."],"tags":["moriio","zmq","protocol-version","kv-transfer","concurrency","vllm"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}