alibaba/canal · error · SocketTimeoutException

Timeout occurred, failed to read total {} bytes in {} millis

Error message

Timeout occurred, failed to read total {} bytes in {} milliseconds, actual read only {} bytes

What it means

Thrown by read(int,int) when the accumulated wait reaches the requested timeout without reading all bytes (remain > 0 && accTimeout >= timeout). The message reports readSize, the budget in ms, and how many bytes were actually read. The loop accumulates SO_TIMEOUT (1s) increments per SocketTimeoutException catch, so the budget is consumed in roughly 1-second granularity.

Source

Thrown at driver/src/main/java/com/alibaba/otter/canal/parse/driver/mysql/socket/BioSocketChannel.java:95

            throw new SocketException("Socket already closed.");
        }
        while (remain > 0 && accTimeout < timeout) {
            try {
                int read = input.read(data, readSize - remain, remain);
                if (read > -1) {
                    remain -= read;
                } else {
                    throw new IOException("EOF encountered.");
                }
            } catch (SocketTimeoutException te) {
                if (Thread.interrupted()) {
                    throw new ClosedByInterruptException();
                }
                accTimeout += SO_TIMEOUT;
            }
        }
        if (remain > 0 && accTimeout >= timeout) {
            throw new SocketTimeoutException(
                "Timeout occurred, failed to read total " + readSize + " bytes in " + timeout
                                             + " milliseconds, actual read only " + (readSize - remain) + " bytes");
        }
        return data;
    }

    @Override
    public void read(byte[] data, int off, int len, int timeout) throws IOException {
        InputStream input = this.input;
        int accTimeout = 0;
        if (input == null) {
            throw new SocketException("Socket already closed.");
        }

        int n = 0;
        while (n < len && accTimeout < timeout) {
            try {
                int read = input.read(data, off + n, len - n);

View on GitHub (pinned to 87be50e876)

Solutions

1. Increase the read timeout budget passed to read(readSize, timeout) for large payloads.
2. Scale the timeout with the expected payload size (the Netty path does this; the BIO path does not).
3. Reduce concurrent load on the MySQL server or move the binlog client closer (network latency).
4. Check for slow queries/long transactions on the master that delay binlog delivery.
5. Verify SO_TIMEOUT granularity (1s) is acceptable for the use case.

Example fix

// before
byte[] b = channel.read(bodyLen, 3000); // too small for big rows

// after
int budget = Math.max(3000, bodyLen / 1024); // ~1ms per KB floor
byte[] b = channel.read(bodyLen, budget);

// choose a budget scaled to payload size before calling read
int budget = Math.max(defaultTimeout, readSize / 1024); // ~1ms/KB floor
byte[] b = channel.read(readSize, budget);

try {
    byte[] b = channel.read(readSize, budget);
} catch (java.net.SocketTimeoutException e) {
    // back off and retry; if persistent, investigate server load/network
}

When it happens

Trigger: Calling read(readSize, timeout) when the server stalls or sends bytes too slowly to fill readSize within timeout milliseconds. Common for large packets or a slow/overloaded MySQL server, or when the network is throttled.

Common situations: Server under heavy load; reading a very large binlog event/transaction; saturated network; the readSize requested is larger than what the server sends before a stall; misconfigured too-small timeout relative to expected packet size.

Understand the failure class

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• Socket already closed.
• EOF encountered.
• socket read timeout occured ! readSize = {}, readableBytes =
• end of stream when reading header
• Start RabbitMQ producer error

AI-assisted analysis of alibaba/canal@87be50e876 (2026-08-14). Data as JSON: /api/errors/a159016f1cee7ab0. Report an issue: GitHub.