{"record":{"id":"c06a6b2f2903d085","repo":"github/copilot-sdk","slug":"llm-inference-response-used-after-rpc-connection-c","errorCode":null,"errorMessage":"LLM inference response used after RPC connection closed","messagePattern":"LLM inference response used after RPC connection closed","errorType":"exception","errorClass":"IOException","httpStatus":null,"severity":"critical","filePath":"java/sdk/src/main/java/com/github/copilot/LlmInferenceExchange.java","lineNumber":250,"sourceCode":"                throw new IOException(\"LLM inference request was cancelled by the runtime\");\n            }\n            if (!started) {\n                throw new IOException(\"LLM inference response writeResponse() called before startResponse()\");\n            }\n            if (finished) {\n                throw new IOException(\n                        \"LLM inference response writeResponse() called after endResponse()/errorResponse()\");\n            }\n        }\n        var params = new LlmInferenceHttpResponseChunkParams(requestId, data, binary ? Boolean.TRUE : null,\n                Boolean.FALSE, null);\n        join(api().httpResponseChunk(params));\n    }\n\n    private ServerLlmInferenceApi api() throws IOException {\n        ServerLlmInferenceApi api = rpcSupplier.get();\n        if (api == null) {\n            throw new IOException(\"LLM inference response used after RPC connection closed\");\n        }\n        return api;\n    }\n\n    private static <T> T join(CompletableFuture<T> future) throws IOException {\n        try {\n            return future.join();\n        } catch (CompletionException | CancellationException e) {\n            Throwable cause = e.getCause() != null ? e.getCause() : e;\n            throw new IOException(cause.getMessage(), cause);\n        }\n    }\n}\n","sourceCodeStart":232,"sourceCodeEnd":264,"githubUrl":"https://github.com/github/copilot-sdk/blob/cd8cf15dc3f9e762615790aaed0a771a0f392755/java/sdk/src/main/java/com/github/copilot/LlmInferenceExchange.java#L232-L264","documentation":"This IOException comes from the private api() accessor of LlmInferenceExchange: the RPC connection to the server has been closed, so rpcSupplier.get() returns null and no further LLM inference API calls can be made. Any exchange operation (startResponse, writeChunk, endResponse, errorResponse) fails because the transport backing the response is gone.","triggerScenarios":"Calling startResponse, writeResponseText/Binary, endResponse, or errorResponse after the underlying RPC connection (rpcSupplier) was torn down — server shutdown, client disconnect, or transport close before the response completed.","commonSituations":"Long-running inference whose stream outlives the connection (client cancelled, proxy timeout); app shutting down while a generation is still streaming; error handling that closes the RPC client in a finally block before draining the exchange.","solutions":["Ensure the RPC connection stays open for the full lifetime of every exchange; only close it after endResponse/errorResponse","Check rpcSupplier/connection state before writing and abort the response gracefully when null","Catch this IOException in streaming code and abort generation instead of retrying writes","Investigate why the connection closed early: timeouts, upstream disconnects, or premature client.close()"],"exampleFix":"// before\nrpcClient.close();\nexchange.writeResponseText(chunk);\n// after\nexchange.writeResponseText(chunk);\nexchange.endResponse();\nrpcClient.close();","handlingStrategy":"try-catch","validationCode":"// cannot query rpc state publicly; keep your own flag\nif (rpcConnectionClosed) return;","typeGuard":null,"tryCatchPattern":"try {\n    exchange.writeResponseText(chunk);\n} catch (IOException e) {\n    if (String.valueOf(e.getMessage()).contains(\"after RPC connection closed\")) {\n        abortGeneration();\n        return;\n    }\n    throw e;\n}","preventionTips":["Close the RPC client only after all exchanges completed","Abort generation promptly when the connection drops","Avoid holding streams open longer than the connection lifetime","Add integration tests covering disconnect mid-stream"],"tags":["io","rpc","connection","lifecycle"],"backgroundTag":"connection-refused","analyzedSha":"cd8cf15dc3f9e762615790aaed0a771a0f392755","analyzedAt":"2026-09-09T18:32:31.973Z","contentChangedAt":"2026-09-09T18:32:31.973Z","schemaVersion":2},"datasetVersion":"2026-09-15T23:17:13.987Z"}