What causes "LLM inference response already finished" in github/copilot-sdk?

Check the exchange's finished state before invoking any response method; abandon late error handling if finished.

github/copilot-sdk · error · IOException

LLM inference response already finished

Error message

LLM inference response already finished

What it means

LlmInferenceExchange.startResponse() also refuses to start a response after the exchange is already finished; it throws IOException('LLM inference response already finished') when the 'finished' flag is set under the lock. Once a response has fully completed, no further response lifecycle calls are accepted for that exchange.

Solutions

Check the exchange's finished state before invoking any response method; abandon late error handling if finished.
Do not reuse LlmInferenceExchange objects across attempts — create a fresh exchange per request.
Catch this IOException in deferred callbacks and treat it as a no-op (the client already got a complete response).

Example fix

// before
scheduledErrorRetry(exchange); // runs after exchange finished
// after
if (!exchange.isFinished()) {
    scheduledErrorRetry(exchange);
}

Defensive patterns

Strategy: type-guard

Validate before calling

if (exchange.isFinished()) {
    LOG.fine("exchange finished; ignoring late response attempt");
    return;
}

Type guard

boolean responseStillOpen(LlmInferenceExchange x) { return !x.isFinished(); }

Try / catch

try {
    exchange.startResponse(status, text, headers);
} catch (IOException e) {
    LOG.fine("exchange already finished; ignoring late callback");
}

Prevention

Check finished state in deferred/async error callbacks before touching the exchange.
Create a fresh exchange per request attempt; never reuse finished exchanges.
Cancel scheduled timeout/error handlers when the exchange completes normally.

When it happens

Trigger: Calling startResponse() (via streamResponse or finalizeError) on an exchange whose response already finished — e.g. a delayed error handler firing after the body was completed (LlmInferenceExchange.java:185).

Common situations: Asynchronous timeout or error callbacks arriving after the exchange completed normally; retry logic mistakenly reusing a finished exchange object for a second attempt.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

AI-assisted analysis of github/copilot-sdk@cd8cf15dc3 (2026-09-09). Data as JSON: /api/errors/d21659ce7ff51226. Report an issue: GitHub.

Appendix: source

Thrown at java/sdk/src/main/java/com/github/copilot/LlmInferenceExchange.java:185

    byte[] drainBody() throws InterruptedException {
        ByteArrayOutputStream out = new ByteArrayOutputStream();
        BodyFrame frame;
        while ((frame = readFrame()) != null) {
            out.writeBytes(frame.data());
        }
        return out.toByteArray();
    }

    // --- Response emit (driven by the handler) ---

    void startResponse(int status, String statusText, Map<String, List<String>> headers) throws IOException {
        synchronized (lock) {
            if (started) {
                throw new IOException("LLM inference response startResponse() called twice");
            }
            if (finished) {
                throw new IOException("LLM inference response already finished");
            }
            started = true;
        }
        var params = new LlmInferenceHttpResponseStartParams(requestId, (long) status, statusText, headers);
        join(api().httpResponseStart(params));
    }

    void writeResponseText(String text) throws IOException {
        writeChunk(text, false);
    }

    void writeResponseBinary(byte[] data) throws IOException {
        writeChunk(Base64.getEncoder().encodeToString(data), true);
    }

    void writeResponseBinary(byte[] data, int offset, int length) throws IOException {
        ByteBuffer encoded = Base64.getEncoder().encode(ByteBuffer.wrap(data, offset, length));
        writeChunk(new String(encoded.array(), 0, encoded.limit(), StandardCharsets.ISO_8859_1), true);

View on GitHub (pinned to cd8cf15dc3)