apache/cassandra · error · TimeoutException

Command

Error message

Command 

What it means

The timeout-aware FBUtilities.exec variant waits for an external command to finish within a given duration. If the process does not complete in time it is destroyed forcibly and a TimeoutException with this message is thrown. It indicates the external tool was too slow or hung.

Solutions

1. Increase the timeout parameter to a value appropriate for the data size
2. Ensure the external command does not wait on stdin or interactive input
3. Investigate why the tool is slow/hung using the logged partial output
4. Retry with a smaller working set or handle the timeout gracefully in the caller

Example fix

// before: FBUtilities.exec(cmd, timeout, TimeUnit.SECONDS) with timeout = 5
// after:  FBUtilities.exec(cmd, 300, TimeUnit.SECONDS) // allow long-running tool to finish

long expectedMs = estimateRuntime(workSize); if (expectedMs > timeoutMs) throw new IllegalStateException("timeout " + timeoutMs + "ms too small for work size");

try { return FBUtilities.exec(cmd, timeout, TimeUnit.SECONDS); } catch (TimeoutException e) { logger.warn("Command timed out, retrying with longer timeout", e); return FBUtilities.exec(cmd, timeout * 4, TimeUnit.SECONDS); }

When it happens

Trigger: Calling the timeout-based FBUtilities.exec(cmd, timeout, ...) where the spawned command does not exit within the timeout, e.g. an external tool blocking on stdin, deadlocked, or working on data far larger than expected.

Common situations: Slow disks making external tools exceed tight timeouts, tools waiting for input (no stdin closed), hung native helpers, or timeouts configured too aggressively for large datasets.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Related errors

• Exception while executing the command:
• Auth check after connection closed
• CasWriteTimeoutException
• CasWriteTimeoutException
• CasWriteTimeoutException

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/a7501e4e13e813c0. Report an issue: GitHub.

Appendix: source

Thrown at src/java/org/apache/cassandra/utils/FBUtilities.java:1280

                completed = true;
            }
            else
            {
                completed = process.waitFor(timeout.toMillis(), TimeUnit.MILLISECONDS);
            }

            copy(process.getInputStream(), out, outBufSize);
            long outOverflow = process.getInputStream().transferTo(overflowSink);

            copy(process.getErrorStream(), err, errBufSize);
            long errOverflow = process.getErrorStream().transferTo(overflowSink);

            if (!completed)
            {
                process.destroyForcibly();
                logger.error("Command {} did not complete in {}, killed forcibly:\noutput:\n{}\n(truncated {} bytes)\nerror:\n{}\n(truncated {} bytes)",
                             Arrays.toString(cmd), timeout, out.asString(), outOverflow, err.asString(), errOverflow);
                throw new TimeoutException("Command " + Arrays.toString(cmd) + " did not complete in " + timeout);
            }
            int r = process.exitValue();
            if (r != 0)
            {
                logger.error("Command {} failed with exit code {}:\noutput:\n{}\n(truncated {} bytes)\nerror:\n{}\n(truncated {} bytes)",
                             Arrays.toString(cmd), r, out.asString(), outOverflow, err.asString(), errOverflow);
                throw new IOException("Command " + Arrays.toString(cmd) + " failed with exit code " + r);
            }
            return out.asString();
        }
    }

    public static void updateChecksumShort(Checksum checksum, short v)
    {
        checksum.update((v >>> 8) & 0xFF);
        checksum.update((v >>> 0) & 0xFF);
    }

View on GitHub (pinned to 88fd0f6a0e)