{"record":{"id":"5852bfdab7179ec6","repo":"apache/cassandra","slug":"timeout-waiting-to-exeute-waiting-transactions","errorCode":null,"errorMessage":"Timeout waiting to exeute waiting transactions","messagePattern":"Timeout waiting to exeute waiting transactions","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"src/java/org/apache/cassandra/service/accord/AccordService.java","lineNumber":647,"sourceCode":"            }\n            else\n            {\n                replayJournal(minSegments);\n\n                if (getAccord().execute_waiting_on_start)\n                {\n                    logger.info(\"Execute waiting transactions...\");\n                    List<AsyncResult<Void>> results = new ArrayList<>();\n                    node.commandStores().forAllUnsafe(commandStore -> results.add(commandStore.tryToExecuteListeningTxns(false)));\n                    if (!results.isEmpty())\n                    {\n                        Future<?> future = toFuture(AsyncResults.reduce(results, Reduce.toNull()));\n                        long timeoutSeconds = getAccord().execute_waiting_on_start_timeout.toSeconds();\n                        if (timeoutSeconds <= 0) future.awaitUninterruptibly().rethrowIfFailed();\n                        else if (!future.awaitUninterruptibly(timeoutSeconds, SECONDS))\n                        {\n                            if (getAccord().execute_waiting_on_start_fail_on_timeout)\n                                throw new RuntimeException(\"Timeout waiting to exeute waiting transactions\");\n                            logger.warn(\"Timeout waiting to exeute waiting transactions\");\n                        }\n                        else future.rethrowIfFailed();\n                    }\n                }\n            }\n        }\n        finally\n        {\n            node.unsafeSetReplaying(false);\n        }\n        node.commandStores().forAllUnsafe(commandStore -> ((AccordCommandStore)commandStore).ensureDurable());\n    }\n\n    private Long2LongHashMap restoreFromSavePoints(CommandStores commandStores)\n    {\n        Long2LongHashMap result = new Long2LongHashMap(0);\n        Future<List<Map.Entry<Integer, Long>>> future = toFuture(((AccordCommandStores)commandStores).restoreState());","sourceCodeStart":629,"sourceCodeEnd":665,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/service/accord/AccordService.java#L629-L665","documentation":"Thrown by AccordService.localStartup() when transactions queued for execution at node start (waiting transactions) do not finish within accord.journal.execute_waiting_on_start_timeout and accord.journal.execute_waiting_on_start_fail_on_timeout is true. This blocks node startup rather than continuing in a degraded state.","triggerScenarios":"Node startup with pending Accord transactions to execute, where the execution future does not complete within the configured timeout seconds and execute_waiting_on_start_fail_on_timeout=true.","commonSituations":"Node starting under heavy load with a large backlog of waiting Accord transactions, slow storage delaying replay/execution, or a too-short execute_waiting_on_start_timeout value.","solutions":["Increase accord.journal.execute_waiting_on_start_timeout to allow the backlog to drain.","Set accord.journal.execute_waiting_on_start_fail_on_timeout=false to continue startup with a warning instead of failing (a warning is logged either way).","Fix root-cause slowness: disk I/O, GC settings, or reduce the transaction backlog before restarting.","Retry startup during lower-load windows after the backlog shrinks."],"exampleFix":"// before\n-Dcassandra.accord.execute_waiting_on_start_timeout=30s\n// after\n-Dcassandra.accord.execute_waiting_on_start_timeout=300s","handlingStrategy":"retry","validationCode":"// before restart with backlog\nlong backlog = estimateWaitingAccordTxns(node);\nif (backlog * avgExecMillis > executeWaitingOnStartTimeoutMs)\n    raiseTimeoutConfig(backlog);","typeGuard":null,"tryCatchPattern":"catch (RuntimeException e) {\n    if (e.getMessage().contains(\"exeute waiting transactions\")) {\n        increaseExecuteWaitingOnStartTimeout();\n        restartNode();\n    } else throw e;\n}","preventionTips":["Size execute_waiting_on_start_timeout to expected backlog drain time.","Avoid restarting nodes with large pending transaction backlogs during peak load.","Monitor disk I/O and GC; fix slowness that delays startup execution."],"tags":["accord","startup","timeout","transactions","backlog"],"backgroundTag":"startup-timeout","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}