sgl-project/sglang · error · RuntimeError
sendmsg sent bytes, expected
Error message
sendmsg sent {sent} bytes, expected {len(payload)} What it means
Weight-cache transport error: a sendmsg/scm_rights write completed with a partial byte count — 'sent' bytes written versus the payload length expected. Short writes break the length-prefixed protocol; the channel is in an undefined state and should be re-established.
Solutions
- Verify peer liveness before transferring FDs
- Retry the whole transfer after restarting the daemon
- Check for protocol misuse (sending on a closed socket)
Defensive patterns
Strategy: try-catch
Try / catch
try:
_send_fd(sock, index, fd)
except RuntimeError:
restart_daemon_and_retry() Prevention
- Confirm peer is reading before FD transfer
- Handle short-send as fatal for the transfer, retry whole session
When it happens
Trigger: Sending an FD over the weight cache transport when the peer socket is closed or the kernel did a short send.
Common situations: Daemon exited while the client was sending tensors; abrupt socket shutdown during FD transfer.
Related errors
- expected one fd, got
- [IpcModelLoader] Error communicating with daemon at
- received truncated fd header
- [weight_cache: ] quantization method is not verified for…
- Connection closed while reading message header
AI-assisted analysis of sgl-project/sglang@0132848349 (2026-08-28).
Data as JSON: /api/errors/602e6f757efe1dec.
Report an issue: GitHub.
Appendix: source
Thrown at python/sglang/srt/weight_cache/transport.py:35
from .protocol import send_msg
logger = logging.getLogger(__name__)
TORCH_IPC_BACKEND = "torch_ipc"
VMM_FD_BACKEND = "vmm_fd"
_FD_INDEX_STRUCT = struct.Struct("<Q")
def _send_fd(sock: socket.socket, fd: int, index: int) -> None:
payload = _FD_INDEX_STRUCT.pack(index)
fds = array.array("i", [int(fd)])
sent = sock.sendmsg(
[payload], [(socket.SOL_SOCKET, socket.SCM_RIGHTS, fds.tobytes())]
)
if sent != len(payload):
raise RuntimeError(f"sendmsg sent {sent} bytes, expected {len(payload)}")
def _recv_fd(sock: socket.socket) -> Tuple[int, int]:
fd_item_size = array.array("i").itemsize
data, ancdata, _, _ = sock.recvmsg(
_FD_INDEX_STRUCT.size, socket.CMSG_SPACE(fd_item_size)
)
if len(data) != _FD_INDEX_STRUCT.size:
raise RuntimeError(
f"received truncated fd header: {len(data)} < {_FD_INDEX_STRUCT.size}"
)
index = _FD_INDEX_STRUCT.unpack(data)[0]
fds = array.array("i")
for level, cmsg_type, cmsg_data in ancdata:
if level == socket.SOL_SOCKET and cmsg_type == socket.SCM_RIGHTS:
fds.frombytes(cmsg_data[: len(cmsg_data) - (len(cmsg_data) % fd_item_size)])
if len(fds) != 1:
for fd in fds:View on GitHub (pinned to 0132848349)