{"record":{"id":"c9722a3a268a5981","repo":"apache/cassandra","slug":"could-not-replay","errorCode":null,"errorMessage":"Could not replay","messagePattern":"Could not replay","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"src/java/org/apache/cassandra/tcm/RemoteProcessor.java","lineNumber":233,"sourceCode":"        if (waitFor == null)\n            return fetchLogAndWait(new CandidateIterator(candidates(true), false), log);\n\n        Future<ClusterMetadata> cmFuture = null;\n        try\n        {\n            Supplier<Future<ClusterMetadata>> fetchFunction = () -> fetchLogAndWaitInternal(new CandidateIterator(candidates(true), false),\n                                                                                            log);\n\n            cmFuture = EpochAwareDebounce.instance.getAsync(fetchFunction, waitFor);\n            return cmFuture.get(retryPolicy.remainingNanos(), TimeUnit.NANOSECONDS);\n        }\n        catch (InterruptedException e)\n        {\n            throw new RuntimeException(\"Can not replay during shutdown\", e);\n        }\n        catch (ExecutionException | TimeoutException e)\n        {\n            throw new RuntimeException(\"Could not replay\", e);\n        }\n    }\n\n    public static ClusterMetadata fetchLogAndWait(CandidateIterator candidateIterator, LocalLog log)\n    {\n        try\n        {\n            return fetchLogAndWaitInternal(candidateIterator, log).await().get();\n        }\n        catch (InterruptedException | ExecutionException e)\n        {\n            throw new RuntimeException(e);\n        }\n    }\n\n    private static Future<ClusterMetadata> fetchLogAndWaitInternal(CandidateIterator candidates, LocalLog log)\n    {\n        try (Timer.Context ctx = TCMMetrics.instance.fetchCMSLogLatency.time())","sourceCodeStart":215,"sourceCodeEnd":251,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/tcm/RemoteProcessor.java#L215-L251","documentation":"RemoteProcessor.fetchLogAndWait catches ExecutionException and TimeoutException from the debounced replay future and rethrows them as a RuntimeException with message 'Could not replay'. ExecutionException means the underlying replay task itself failed; TimeoutException means the remote fetch/replay did not finish within the retry policy's remaining time budget. Either way the node failed to obtain a consistent ClusterMetadata state from remote peers.","triggerScenarios":"cmFuture.get(...) times out because the retry policy budget (retryPolicy.remainingNanos()) is exhausted before peers respond; the replay computation threw internally (network failure to CMS peers, serialization issue, peer returned an error) and surfaces as ExecutionException.","commonSituations":"CMS members unreachable or slow at node startup; timeouts set too aggressively in the retry policy for a large or WAN-separated cluster; a bug/exception inside the fetch function that is masked until unwrapped here.","solutions":["Inspect the cause chain of this RuntimeException — the ExecutionException/TimeoutException cause names the real failure.","Verify connectivity and health of CMS members (they answer fetch-log requests).","Increase the TCM fetch/replay timeout / retry policy if the cluster is large or network latency is high.","If the cause is a replay-time exception (e.g. deserialization), fix the underlying metadata corruption or version mismatch before retrying.","Restart the node once the CMS quorum is healthy to retry catch-up."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { ClusterMetadata cm = RemoteProcessor.fetchLogAndWait(candidates, log); }\ncatch (RuntimeException e) {\n    Throwable cause = e.getCause();\n    if (cause instanceof TimeoutException) { /* retry with larger budget */ }\n    else if (cause instanceof ExecutionException) { /* inspect cause.getCause() for root failure */ }\n    else throw e;\n}","preventionTips":["Size the retry policy for worst-case network latency","Monitor CMS member health before/while replaying","Always unwrap and log the cause chain","Alert on fetchPeerLogLatency outliers"],"tags":["tcm","timeout","metadata-replay","remote-fetch"],"backgroundTag":"request-timeout","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-14T11:17:12.474Z"}