{"record":{"id":"9ad5159c82796623","repo":"apache/cassandra","slug":"commits-are-paused-not-trying-to-commit","errorCode":null,"errorMessage":"Commits are paused, not trying to commit ","messagePattern":"Commits are paused, not trying to commit ","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"src/java/org/apache/cassandra/tcm/ClusterMetadataService.java","lineNumber":683,"sourceCode":"     *   - Sends the commit with a TCM_COMMIT_REQ message\n     *   - Message expires after min(cms_await_timeout, remaining sender deadline) to account for CMS node failures.\n     *   - On failure/timeout retries use exponential backoff with full jitter to decorrelate cms await timeouts.\n     * For CMS members (LOCAL state / AbstractLocalProcessor) for local TCM commits:\n     *   - The main outer retry policy described below for TCM_COMMIT_REQ is used without the message expiry,\n     *     as the commit runs locally through Paxos without a remote message hop.\n     * For CMS members handling TCM_COMMIT_REQ messages (Commit.Handler):\n     *   - Deadline is max(now + write_rpc_timeout, message.expiresAtNanos() - write_rpc_timeout), with exponential\n     *     backoff and full jitter using the TCM admin initial/max delay. The floor guarantees at least one attempt\n     *     window even when cms_await_timeout is misconfigured close to write_rpc_timeout.\n     *   - The CMS member reduces its own retry deadline by write_rpc_timeout before the message expiry so it exhausts\n     *     retries and returns an explicit failure before the sender's per-message callback fires (see Commit.Handler).\n     *\n     * @param onFailure     handler checks if rejection has resulted from a retry of the same trasformation.\n     */\n    public <T1> T1 commit(Transformation transform, CommitSuccessHandler<T1> onSuccess, CommitFailureHandler<T1> onFailure)\n    {\n        if (commitsPaused.get())\n            throw new IllegalStateException(\"Commits are paused, not trying to commit \" + transform);\n\n        long startTime = nanoTime();\n        // Replay everything in-flight before attempting a commit\n        // We grab highest consecutive epoch here, since we want both local and remote processors to benefit from\n        // discover-own-commits via entry id in case of lost messages (in remote case) and Paxos re-proposals (in local case)\n        Epoch highestConsecutive = log.waitForHighestConsecutive().epoch;\n\n        Retry retryPolicy = getRetryPolicy(transform.kind());\n        logger.info(\"Committing {} with {}\", transform.kind(), retryPolicy);\n        Commit.Result result = processor.commit(entryIdGen.get(), transform, highestConsecutive, retryPolicy);\n\n        try\n        {\n            if (result.isSuccess())\n            {\n                TCMMetrics.instance.commitSuccessLatency.update(nanoTime() - startTime, NANOSECONDS);\n                return onSuccess.accept(awaitAtLeast(result.success().epoch));\n            }","sourceCodeStart":665,"sourceCodeEnd":701,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/tcm/ClusterMetadataService.java#L665-L701","documentation":"ClusterMetadataService supports pausing metadata commits (e.g. during shutdown, reconfiguration, or migration). If commitsPaused is set, the two-argument commit(transform, onSuccess, onFailure) immediately throws IllegalStateException instead of queueing, signaling the caller that the transformation was never attempted because commits are globally suspended.","triggerScenarios":"Calling commit() (including forceSnapshot) while ClusterMetadataService.commitsPaused AtomicBoolean is true — typically set during node shutdown, TCM migration, or administrative pause of metadata changes.","commonSituations":"Schema or topology change issued while the node is shutting down or restarting; management tooling pauses commits for maintenance but a background task still attempts a commit; migration scripts issuing commits after pausing.","solutions":["Ensure commits are resumed (commitsPaused=false) before issuing transformations.","Move the commit attempt before the pause window or after resume completes.","Retry the operation once commits are unpaused; the transformation was not attempted so retry is safe.","Fix automation to check pause state before issuing schema/topology operations during maintenance."],"exampleFix":"// before\nservice.commit(new AlterTable(...)); // throws while commits paused\n// after\nif (!service.isCommitsPaused())\n    service.commit(new AlterTable(...));\nelse\n    logger.warn(\"Deferring commit; commits are paused\");","handlingStrategy":"validation","validationCode":"if (ClusterMetadataService.instance().isCommitsPaused()) deferCommit(transform);","typeGuard":null,"tryCatchPattern":"try { service.commit(transform); } catch (IllegalStateException e) { if (e.getMessage().startsWith(\"Commits are paused\")) scheduleRetryAfterResume(); else throw e; }","preventionTips":["Do not issue schema/topology changes during maintenance windows with paused commits","Check pause state in automation before committing","Retry after resume — the transformation was never attempted"],"tags":["commit","paused","cluster-metadata","shutdown"],"backgroundTag":"invalid-state-transition","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}