{"record":{"id":"5350bffb088d3cc6","repo":"sgl-project/sglang","slug":"rpc-error-on-self-debug-name-response-error","errorCode":null,"errorMessage":"RPC error on {self._debug_name}: {response['error']}","messagePattern":"RPC error on (.+?): (.+?)","errorType":"exception","errorClass":"RuntimeError","httpStatus":null,"severity":"error","filePath":"python/sglang/srt/debug_utils/dumper.py","lineNumber":1502,"sourceCode":"    def __init__(self, socket, debug_name: str):\n        self._socket = socket\n        self._debug_name = debug_name\n\n    def __getattr__(self, method_name: str):\n        def call(*args, **kwargs):\n            sock_send(\n                self._socket,\n                wrap_as_pickle(\n                    {\n                        \"method\": method_name,\n                        \"args\": args,\n                        \"kwargs\": kwargs,\n                    }\n                ),\n            )\n            response = sock_recv(self._socket)\n            if response[\"error\"]:\n                raise RuntimeError(\n                    f\"RPC error on {self._debug_name}: {response['error']}\"\n                )\n            return response[\"result\"]\n\n        return call\n\n\nclass _RpcBroadcastBase:\n    \"\"\"Base for broadcasting method calls to dumper instance(s).\"\"\"\n\n    def __getattr__(self, method_name: str):\n        raise NotImplementedError\n\n    def __init__(self, handles: List[_ZmqRpcHandle]):\n        self._handles = handles\n\n\nclass _ZmqRpcBroadcast(_RpcBroadcastBase):","sourceCodeStart":1484,"sourceCodeEnd":1520,"githubUrl":"https://github.com/sgl-project/sglang/blob/0132848349585cfe6aae51c4941cbae872505f8a/python/sglang/srt/debug_utils/dumper.py#L1484-L1520","documentation":"Raised by the debug-utils dumper RPC client when the remote side returns a response containing a non-null 'error' field. It wraps the remote error message with the debug name of the RPC endpoint so you can tell which service failed. It is a transport-level wrapper: the root cause text comes from whatever the serving process returned.","triggerScenarios":"Calling a method on an RPC proxy object created by this dumper module (e.g. proxy.fn(...)) where the remote handler raised an exception; the remote exception is serialized into response['error'] and re-raised locally as RuntimeError. Also triggered if the socket connection carries a malformed/stale response from a previous call.","commonSituations":"The debugged process crashed or raised inside the RPC handler (e.g. CUDA error, invalid tensor op); attaching to a dead or restarted process while reusing the old socket; calling an RPC method with invalid arguments for the remote API version.","solutions":["Inspect the text after 'RPC error on <name>:' — it contains the remote traceback; fix the underlying exception on the server side","Verify the remote process is still alive and the socket was not recycled (reconnect if the pid restarted)","Check that arguments you pass match what the remote handler expects in this sglang version","If the error is intermittent, serialize calls or re-establish the RPC connection before retrying"],"exampleFix":"// before\nresult = proxy.capture_tensors()\n\n// after\ntry:\n    result = proxy.capture_tensors()\nexcept RuntimeError as e:\n    if 'RPC error on' in str(e):\n        print('remote failed:', e)\n        proxy = reconnect()\n        result = proxy.capture_tensors()\n    else:\n        raise","handlingStrategy":"try-catch","validationCode":"result = proxy.ping()  # cheap liveness RPC before the real call","typeGuard":null,"tryCatchPattern":"try:\n    result = proxy.call(...)\nexcept RuntimeError as e:\n    if 'RPC error on' not in str(e):\n        raise\n    log_remote_failure(e)\n    proxy = reconnect()\n    result = retry_call(proxy)","preventionTips":["Check remote process liveness before issuing RPC calls","Always log the full wrapped message — the remote traceback is embedded in it","Wrap RPC calls in a helper that reconnects once on failure"],"tags":["rpc","debug-utils","remote-error","sglang"],"backgroundTag":"remote-procedure-call-failed","analyzedSha":"0132848349585cfe6aae51c4941cbae872505f8a","analyzedAt":"2026-08-28T05:10:05.995Z","schemaVersion":2},"datasetVersion":"2026-08-28T06:17:29.519Z"}