github/copilot-sdk · error · InvalidOperationException

LLM inference response used after RPC connection closed.

Error message

LLM inference response used after RPC connection closed.

What it means

LlmInferenceExchange sends each response chunk over the server RPC channel obtained via a lazy _getServerRpc accessor. When the RPC connection has been closed the accessor returns null, and this InvalidOperationException is thrown instead of attempting an RPC call on a dead channel.

Solutions

1. Discard/abort the exchange when the RPC connection closes rather than writing chunks
2. Track connection lifetime and cancel in-flight exchanges on disconnect
3. Re-create the exchange and retry over a fresh connection if the request is retriable
4. Check for exceptions during streaming and clean up exchange references in finally blocks

Example fix

// before
await exchange.WriteAsync(chunk); // may throw if RPC closed
// after
if (!connection.IsOpen) { await exchange.DisposeAsync(); return; }
await exchange.WriteAsync(chunk);

if (_getServerRpc() is null) { await exchange.DisposeAsync(); return; }

bool RpcAlive(Func<ServerRpc?> getRpc) => getRpc() is not null;

try { await exchange.WriteAsync(chunk); }
catch (InvalidOperationException ex) when (ex.Message.Contains("RPC connection closed")) { /* abort response, clean up exchange */ }

When it happens

Trigger: Writing or finishing a response chunk after the underlying RPC connection was closed/disposed — e.g. the runtime disconnected mid-response, or the exchange outlived the connection scope.

Common situations: Long-running LLM generation outlives a dropped connection; server shutdown closes the RPC while a response is still streaming; the exchange is cached and reused after reconnect.

Understand the failure class

• Connection failures: ECONNREFUSED, ECONNRESET, and friends — why connections get refused, reset, or dropped.

Related errors

• Cannot connect because TCP host or port are not available
• Failed to delete session
• LLM inference request was cancelled by the runtime.
• LLM inference response StartAsync() called twice.
• LLM inference response used after RPC connection closed

AI-assisted analysis of github/copilot-sdk@cd8cf15dc3 (2026-09-09). Data as JSON: /api/errors/bf64e4dc8f668308. Report an issue: GitHub.

Appendix: source

Thrown at dotnet/src/CopilotRequestHandler.cs:834

        }

        if (!_started)
        {
            throw new InvalidOperationException("LLM inference response WriteAsync() called before StartAsync().");
        }

        if (_finished)
        {
            throw new InvalidOperationException("LLM inference response WriteAsync() called after EndAsync()/ErrorAsync().");
        }

        await ServerRpc()
            .LlmInference.HttpResponseChunkAsync(RequestId, data, binary: binary, end: false)
            .ConfigureAwait(false);
    }

    private ServerRpc ServerRpc() =>
        _getServerRpc() ?? throw new InvalidOperationException("LLM inference response used after RPC connection closed.");

    private static Dictionary<string, IList<string>> ToWireHeaders(IReadOnlyDictionary<string, IReadOnlyList<string>>? headers)
    {
        var result = new Dictionary<string, IList<string>>(StringComparer.OrdinalIgnoreCase);
        if (headers is null)
        {
            return result;
        }

        foreach (var (name, values) in headers)
        {
            result[name] = values as IList<string> ?? [.. values];
        }

        return result;
    }

    private struct BodyItem

View on GitHub (pinned to cd8cf15dc3)