apache/cassandra · error · RuntimeException

Timeout - task did not finish in

Error message

Timeout - task did not finish in ${timeout}

What it means

FBUtilities.waitOnFuture blocks on an Awaitable/Future with a caller-supplied timeout. If the task does not complete within `timeout` units, the TimeoutException is rethrown as a plain RuntimeException with a message naming the timeout. It signals that an internal async task (e.g. schema pull,bootstrap step) exceeded its deadline.

Solutions

1. Check the awaited node is alive and responsive (nodetool status, logs); restart or replace it
2. Increase the timeout argument at the call site if the operation is legitimately slow
3. Inspect why the underlying task hangs (thread dumps, tracing) and fix the root cause
4. Retry the operation once the remote node recovers

Example fix

// before
FBUtilities.waitOnFuture(future, 5, TimeUnit.SECONDS);
// after
FBUtilities.waitOnFuture(future, 60, TimeUnit.SECONDS); // allow slow large task to finish

try {
    FBUtilities.waitOnFuture(future, timeout, TimeUnit.SECONDS);
} catch (RuntimeException e) {
    if (e.getMessage() != null && e.getMessage().startsWith("Timeout - task did not finish")) {
        // inspect peer health, retry with larger timeout or fail the operation
    } else throw e;
}

When it happens

Trigger: Calling FBUtilities.waitOnFuture(future, timeout) where the future's computation is still pending when the deadline expires; e.g. waiting on schema agreement, REMOTE tasks via StageManager, or Verbs-based one-shot tasks whose peer is slow or dead.

Common situations: A target node is overloaded, GC-paused, down, or network-partitioned so the awaited remote task never returns; timeout configured too aggressively; bootstrap/repair hanging on a large operation.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• Auth check after connection closed
• CasWriteTimeoutException
• CasWriteTimeoutException
• CasWriteTimeoutException
• CasWriteUnknownResultException

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/d9e388475537a569. Report an issue: GitHub.

Appendix: source

Thrown at src/java/org/apache/cassandra/utils/FBUtilities.java:596

    public static <T> T waitOnFuture(Future<T> future, Duration timeout)
    {
        Preconditions.checkArgument(!timeout.isNegative(), "Timeout must not be negative, provided %s", timeout);
        try
        {
            return future.get(timeout.toNanos(), TimeUnit.NANOSECONDS);
        }
        catch (ExecutionException ee)
        {
            logger.info("Exception occurred in async code", ee);
            throw Throwables.cleaned(ee);
        }
        catch (InterruptedException ie)
        {
            throw new AssertionError(ie);
        }
        catch (TimeoutException e)
        {
            throw new RuntimeException("Timeout - task did not finish in " + timeout);
        }
    }

    public static <T, F extends Future<? extends T>> F waitOnFirstFuture(Iterable<? extends F> futures)
    {
        return waitOnFirstFuture(futures, 100);
    }
    /**
     * Only wait for the first future to finish from a list of futures. Will block until at least 1 future finishes.
     * @param futures The futures to wait on
     * @return future that completed.
     */
    public static <T, F extends Future<? extends T>> F waitOnFirstFuture(Iterable<? extends F> futures, long delay)
    {
        while (true)
        {
            Iterator<? extends F> iter = futures.iterator();
            if (!iter.hasNext())

View on GitHub (pinned to 88fd0f6a0e)