{"record":{"id":"c2d1e2f6579bf35d","repo":"vllm-project/vllm","slug":"json-error-0","errorCode":null,"errorMessage":"JSON error: {0}","messagePattern":"JSON error: (.+?)","errorType":"exception","errorClass":"BenchError","httpStatus":null,"severity":"critical","filePath":"rust/src/bench/src/error.rs","lineNumber":11,"sourceCode":"// SPDX-License-Identifier: Apache-2.0\n// SPDX-FileCopyrightText: Copyright contributors to the vLLM project\n\nuse thiserror::Error;\n\n#[derive(Error, Debug)]\npub enum BenchError {\n    #[error(\"HTTP request failed: {0}\")]\n    Http(#[from] reqwest::Error),\n\n    #[error(\"JSON error: {0}\")]\n    Json(#[from] serde_json::Error),\n\n    #[error(\"Tokenizer error: {0}\")]\n    Tokenizer(String),\n\n    /// The server's /tokenize//detokenize endpoint is not usable (4xx status:\n    /// not exposed, or rejected by a gateway such as LLM-d/EPP that returns\n    /// 400 instead of 404). Callers treat this as \"skip verification\", unlike\n    /// `Tokenizer` errors which are genuine failures.\n    #[error(\"tokenize endpoint unavailable: {0}\")]\n    TokenizeUnavailable(String),\n\n    #[error(\"Configuration error: {0}\")]\n    Config(String),\n\n    #[error(\"Endpoint not ready after {0}s: {1}\")]\n    EndpointTimeout(u64, String),\n","sourceCodeStart":1,"sourceCodeEnd":29,"githubUrl":"https://github.com/vllm-project/vllm/blob/c794754062d49a8fdb63ab3c5215b488b865030c/rust/src/bench/src/error.rs#L1-L29","documentation":"Raised by MoRIIOWrapper.register_local_tensor (moriio_engine.py:604) when the underlying MoRIIO engine fails to register a torch.Tensor as RDMA-accessible local memory, or when register_torch_tensor returns None. Registration pins/maps the tensor's storage so a remote node can RDMA read/write it; the resulting MemoryDesc is packed and exchanged with the peer to build the RDMA session. Any exception from the engine (invalid tensor, non-CUDA/non-pinned memory, OOM during registration, IBV registration failure) is wrapped in MoRIIOError with the cause chained via 'from e'.","triggerScenarios":"Calling register_local_tensor(kv_cache) as the connector does at moriio_connector.py:1775 during KV-role startup, where the tensor is not eligible for RDMA registration: CPU tensor that is not pinned, a non-contiguous or zero-element tensor, a view whose storage the NIC cannot register, CUDA memory on a device invisible to the RDMA stack, or ibv_reg_mr failure from an IOMMU/vfio mismatch or MR limit exhaustion. Also triggered when register_torch_tensor returns None, which the assert converts into the same MoRIIOError.","commonSituations":"Running MoRIIO disaggregated prefill/decode on nodes where the GPU is not on the same NUMA/IOMMU domain as the RDMA NIC; CUDA_VISIBLE_DEVICES hiding the GPU the NIC's GID refers to; kv cache tensors created with torch.empty on CPU without page-locked memory in CPU-offload setups; older MoRIIO builds whose register_torch_tensor returns None instead of raising; exceeding the NIC's memory-region limit after many engine restarts in one process.","solutions":["Read the chained cause: the '{e}' text names the real failure — act on that message first (e.g. 'failed to register memory region', 'invalid device', 'None returned').","Verify the tensor being registered is the real CUDA KV cache tensor: .is_cuda, .is_contiguous(), .numel() > 0, and lives on the device the MoRIIO engine was initialized with.","Check RDMA stack health on the node: the NIC is up, its GID routes to the GPU (nvidia-smi topo -m, ibstat), and CUDA_VISIBLE_DEVICES does not remap device indices unexpectedly.","If registration fails with MR/resource errors, restart the engine process (leaked registrations from earlier attempts hold MRs) and reduce number of registered tensors.","Upgrade/align the MoRIIO package version so register_torch_tensor either raises a descriptive error or never returns None."],"exampleFix":"// before\npacked = wrapper.register_local_tensor(kv_cache)  # may wrap opaque engine errors\n\n// after (validate eligibility first, then surface the chained cause)\nassert kv_cache.is_cuda and kv_cache.is_contiguous() and kv_cache.numel() > 0, (\n    f\"kv_cache not registrable: cuda={kv_cache.is_cuda} \"\n    f\"contiguous={kv_cache.is_contiguous()} numel={kv_cache.numel()}\"\n)\ntry:\n    packed = wrapper.register_local_tensor(kv_cache)\nexcept MoRIIOError as e:\n    raise RuntimeError(\n        f\"RDMA registration of KV cache failed on device \"\n        f\"{kv_cache.device}: {e.__cause__ or e}\"\n    ) from e","handlingStrategy":"validation","validationCode":"def is_registrable_kv_cache(tensor) -> bool:\n    return (\n        tensor.is_cuda\n        and tensor.is_contiguous()\n        and tensor.numel() > 0\n        and tensor.data_ptr() != 0\n    )\n\n# before engine startup / registration:\nassert is_registrable_kv_cache(kv_cache), (\n    f\"KV cache not RDMA-registrable: cuda={kv_cache.is_cuda}, \"\n    f\"contig={kv_cache.is_contiguous()}, numel={kv_cache.numel()}\"\n)\nwrapper.register_local_tensor(kv_cache)","typeGuard":"from typing import TypeGuard\nimport torch\n\ndef is_cuda_kv_cache_tensor(t: object) -> TypeGuard[torch.Tensor]:\n    return (\n        isinstance(t, torch.Tensor)\n        and t.is_cuda\n        and t.is_contiguous()\n        and t.numel() > 0\n        and not t.requires_grad\n    )","tryCatchPattern":"from vllm.distributed.kv_transfer.kv_connector.v1.moriio.moriio_engine import MoRIIOError\n\ntry:\n    packed_meta = wrapper.register_local_tensor(kv_cache)\nexcept MoRIIOError as e:\n    cause = e.__cause__ or e\n    logger.error(\"RDMA registration failed on %s: %s\", kv_cache.device, cause)\n    raise RuntimeError(\n        \"KV cache RDMA registration failed; check GPU/NIC affinity \"\n        \"(nvidia-smi topo -m) and MoRIIO engine init\"\n    ) from e","preventionTips":["Register only the real CUDA-allocated KV cache tensor; never a CPU tensor, a view with odd strides, or an empty tensor.","Do registration exactly once per KV cache allocation; re-register after any resize rather than leaking memory regions.","Validate GPU/NIC topology (same PCIe root complex / NUMA node) before starting MoRIIO engines.","Pin the MoRIIO package version across all prefill and decode nodes so register_torch_tensor's contract (raise vs return None) is consistent."],"tags":["moriio","rdma","kv-transfer","memory-registration","vllm"],"backgroundTag":null,"analyzedSha":"c794754062d49a8fdb63ab3c5215b488b865030c","analyzedAt":"2026-08-14T21:17:39.825Z","schemaVersion":2},"datasetVersion":"2026-08-15T17:31:12.345Z"}