apache/incubator-seata · error · RuntimeException

Stream closed unexpectedly

Error message

Stream closed unexpectedly

What it means

Thrown from SeataHttpWatch.next() when the underlying BufferedSource.readUtf8Line() returns null, meaning the server closed the event stream (EOF) while the client was still iterating events. Because the watch protocol expects an unbounded stream of prefixed lines, EOF is treated as an unexpected termination rather than normal completion.

Source

Thrown at common/src/main/java/org/apache/seata/common/util/SeataHttpWatch.java:165

            // Check if source is exhausted (stream closed)
            return !source.exhausted();
        } catch (IOException e) {
            LOGGER.error("Error checking if stream has more data", e);
            return false;
        }
    }

    @Override
    public Response<T> next() {
        try {
            /*
            Read a single line and parse it as an event.
            Format: "{prefix}{json}\n" where prefix is defined in Constants.WATCH_EVENT_PREFIX.
            Each line is a complete event, event type is included in the JSON data.
            */
            String line = source.readUtf8Line();
            if (line == null) {
                throw new RuntimeException("Stream closed unexpectedly");
            }

            if (!line.startsWith(Constants.WATCH_EVENT_PREFIX)) {
                throw new RuntimeException("Invalid event format: expected prefix '" + Constants.WATCH_EVENT_PREFIX
                        + "', got: " + (line.length() > 20 ? line.substring(0, 20) + "..." : line));
            }

            String jsonData = line.substring(Constants.WATCH_EVENT_PREFIX.length());
            return parseEvent(jsonData);

        } catch (IOException e) {
            throw new RuntimeException("IO Exception during next()", e);
        }
    }

    /**
     * Parse event JSON into Response object.
     * Simplified format: only contains group, timestamp, and metadata fields.

View on GitHub (pinned to e01f97c6db)

Solutions

1. Treat this as a signal to reconnect: create a new watch via createWatch and resume iteration.
2. If a proxy/load balancer terminates idle SSE connections, raise its read timeout or enable keepalive heartbeats on the server side.
3. Check whether the Seata server restarted (server logs) at the moment of the failure.
4. Pin down network instability between client and server if restarts are not the cause.

Example fix

// before
while (true) {
    Response<MyEvent> ev = watch.next();
    handle(ev);
}

// after: reconnect on stream end
while (running) {
    try (SeataHttpWatch<MyEvent> w = SeataHttpWatch.createWatch(call.clone(), MyEvent.class)) {
        while (true) {
            handle(w.next());
        }
    } catch (RuntimeException e) {
        // 'Stream closed unexpectedly' or IO failure -> reconnect with backoff
        sleepBackoff();
    }
}

catch (RuntimeException e) {
    if ("Stream closed unexpectedly".equals(e.getMessage())) {
        watch = SeataHttpWatch.createWatch(call.clone(), eventType); // reconnect
        continue;
    }
    throw e;
}

When it happens

Trigger: Iterating watch.next() and the server closes the connection: server shutdown/restart, idle-timeout enforced by a proxy or load balancer, network drop, or the server deliberately ending the stream after an error event.

Common situations: Long-lived watch connections killed by LB idle timeouts (default nginx 60s read timeout with no keepalive events); Seata server rolling deployment; flaky network between client and server; watchdog code that assumes the stream never ends.

Related errors

• Watch request failed: {}
• IO Exception during next()
• can not connect to [{invalidAddress}]
• ip and port string cannot be empty!
• "Invalid endpoint format: " + address + ". Endpoint should b

AI-assisted analysis of apache/incubator-seata@e01f97c6db (2026-08-14). Data as JSON: /api/errors/1390e0a286fc62ce. Report an issue: GitHub.