apache/incubator-seata · error · TimeoutException

%s ,cost: %d ms

Error message

%s ,cost: %d ms

What it means

MessageFuture.get(timeout, unit) is the sync-RPC primitive in Seata remoting: the caller blocks until the response (or an exception placeholder) completes the future. On timeout it rethrows TimeoutException with this message, appending the original timeout text and the measured elapsed milliseconds (System.currentTimeMillis() - start) so operators can compare requested vs actual wait.

Source

Thrown at core/src/main/java/org/apache/seata/core/protocol/MessageFuture.java:64

     * Get object.
     *
     * @param timeout the timeout
     * @param unit    the unit
     * @return the object
     * @throws TimeoutException the timeout exception
     * @throws InterruptedException the interrupted exception
     */
    public Object get(long timeout, TimeUnit unit) throws TimeoutException, InterruptedException {
        Object result = null;
        try {
            result = origin.get(timeout, unit);
            if (result instanceof TimeoutException) {
                throw (TimeoutException) result;
            }
        } catch (ExecutionException e) {
            throw new ShouldNeverHappenException("Should not get results in a multi-threaded environment", e);
        } catch (TimeoutException e) {
            throw new TimeoutException(
                    String.format("%s ,cost: %d ms", e.getMessage(), System.currentTimeMillis() - start));
        }

        if (result instanceof RuntimeException) {
            throw (RuntimeException) result;
        } else if (result instanceof Throwable) {
            throw new RuntimeException((Throwable) result);
        }

        return result;
    }

    /**
     * Sets result message.
     *
     * @param obj the obj
     */
    public void setResultMessage(Object obj) {

View on GitHub (pinned to e01f97c6db)

Solutions

1. Raise the client RPC timeouts (e.g. seata.client.tm.commit-retry-timeout / default-transaction-group timeout, rm/tm rpc timeout in ms) to comfortably exceed observed p99 round-trips.
2. Fix TC-side latency: tune the store (db pool, index on global/branch table), check GC logs, give TC adequate CPU.
3. Verify network RTT and MTU between client and TC; eliminate lossy links or saturated NICs.
4. Ensure the transaction group -> TC endpoint mapping is correct so requests are not silently routed to a dead node.

Example fix

# before
tm.degrade-check=false
default-global-transaction-timeout=60000
# after: raise rpc timeout and fix tc store latency
tm.default-global-transaction-timeout=120000
# plus: dbstore connection pool sizing on tc

// preflight: confirm TC reachable and rpc timeout comfortably above RTT p99 before beginning the global tx
if (!tcHealthCheck(1000)) { deferOrRouteToHealthyTC(); }
assert configuredRpcTimeoutMillis > 5 * measuredP99Rtt : "rpc timeout too tight";

catch (TimeoutException e) {
    long cost = parseCost(e.getMessage());
    if (cost >= configuredTimeout) { // genuine timeout: retry idempotent ops once with backoff
        retryWithBackoff(this::idempotentRpc, 2, 500);
    } else { escalateToInfra(e); }
}

When it happens

Trigger: sendSync-style calls (TM begin/commit, RM register/undo-report) where the peer does not answer within timeoutMillis: TC busy in GC or long store operations, network latency, or the response lost after a connection reset.

Common situations: Default RPC timeout too low for the deployment (store.db slow queries on TC), TC paused by full GC, k8s CPU throttling of the TC pod, or bursty load making commit round-trips exceed the configured timeout.

Related errors

• "invalid address:" + address
• IO Exception during next()
• MCP PUT Call TC Failed.
• timeout should more than 0ms
• connect failed, can not connect to services-server.

AI-assisted analysis of apache/incubator-seata@e01f97c6db (2026-08-14). Data as JSON: /api/errors/589a24fec274f263. Report an issue: GitHub.