{"record":{"id":"b91eaac2ab50d37d","repo":"apache/cassandra","slug":"couldn-t-commit-the-transformation-is-the-node-sh","errorCode":null,"errorMessage":"Couldn't commit the transformation. Is the node shutting down?","messagePattern":"Couldn't commit the transformation\\. Is the node shutting down\\?","errorType":"exception","errorClass":"IllegalStateException","httpStatus":null,"severity":"error","filePath":"src/java/org/apache/cassandra/tcm/ClusterMetadataService.java","lineNumber":717,"sourceCode":"                TCMMetrics.instance.commitSuccessLatency.update(nanoTime() - startTime, NANOSECONDS);\n                return onSuccess.accept(awaitAtLeast(result.success().epoch));\n            }\n            else\n            {\n                TCMMetrics.instance.recordCommitFailureLatency(nanoTime() - startTime, NANOSECONDS, result.failure().rejected);\n                logger.debug(\"Failed to commit {} after {} attempts ({}): {} {}\",\n                             transform.kind(), retryPolicy.attempts(), retryPolicy,\n                             result.failure().code, result.failure().message);\n                return onFailure.accept(result.failure().code, result.failure().message);\n            }\n        }\n        catch (TimeoutException t)\n        {\n            throw new IllegalStateException(String.format(\"Timed out while waiting for the follower to enact the epoch %s\", result.success().epoch), t);\n        }\n        catch (InterruptedException e)\n        {\n            throw new IllegalStateException(\"Couldn't commit the transformation. Is the node shutting down?\", e);\n        }\n    }\n\n    private static Retry getRetryPolicy(Transformation.Kind kind)\n    {\n        Retry retryPolicy;\n        if (kind == Transformation.Kind.STARTUP)\n        {\n            retryPolicy = Retry.withNoTimeLimit(TCMMetrics.instance.commitRetries, Retry.unsafeRetryIndefinitely());\n        }\n        else if (kind == Transformation.Kind.SCHEMA_CHANGE)\n        {\n            long deadlineNanos = nanoTime() + DatabaseDescriptor.getRpcTimeout(TimeUnit.NANOSECONDS);\n            retryPolicy = Retry.until(deadlineNanos, TCMMetrics.instance.commitRetries);\n        }\n        else\n        {\n            // On non-CMS members, which send commit requests via messaging to the CMS members, the exponential backoff","sourceCodeStart":699,"sourceCodeEnd":735,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/tcm/ClusterMetadataService.java#L699-L735","documentation":"ClusterMetadataService.commit() blocks waiting for the committed transformation's epoch to be enacted locally. The wait can be interrupted, and this library converts that InterruptedException into an IllegalStateException with this message, preserving the cause. It signals that the commit did complete (or fail) but the caller thread's wait was interrupted — most commonly during node shutdown.","triggerScenarios":"Calling ClusterMetadataService.commit(transform, onSuccess, onFailure) from a thread that gets interrupted while waiting in awaitAtLeast(result.success().epoch) after a successful commit; typical when the node is draining/shutting down and the calling thread is interrupted.","commonSituations":"Node shutdown or drain while schema changes or topology changes (e.g. REMOVENODE, bootstrap) are being committed; test teardown interrupting threads; executors cancelled mid-commit.","solutions":["Retry the operation after ensuring the node is not shutting down; check StorageService operation mode before committing","Do not swallow InterruptedException — if you wrap the commit call, re-interrupt the thread (Thread.currentThread().interrupt()) before retrying","Re-run the transformation after restart; TCM log commits are retried safely since transformation state is persisted in the cluster metadata log","If seen during startup/shutdown races in tests, use test framework hooks to await node readiness before committing"],"exampleFix":"// before\nClusterMetadataService.instance.commit(transform, ok -> ok, (c, m) -> { throw new IllegalStateException(m); });\n// after\nif (!StorageService.instance.isStarting() && !StorageService.instance.isShutdown())\n{\n    try\n    {\n        ClusterMetadataService.instance.commit(transform, ok -> ok, (c, m) -> { throw new IllegalStateException(m); });\n    }\n    catch (IllegalStateException e)\n    {\n        if (e.getCause() instanceof InterruptedException)\n            Thread.currentThread().interrupt(); // re-interrupt before retry/abort\n        throw e;\n    }\n}","handlingStrategy":"try-catch","validationCode":"if (StorageService.instance.isShutdown() || StorageService.instance.isStarting())\n    throw new IllegalStateException(\"Node is shutting down/startup; skip commit\");","typeGuard":"boolean canCommit = !Thread.currentThread().isInterrupted()\n                   && !StorageService.instance.isShutdown();","tryCatchPattern":"try\n{\n    ClusterMetadataService.instance.commit(transform, ok -> ok, (c, m) -> { throw new IllegalStateException(m); });\n}\ncatch (IllegalStateException e)\n{\n    if (e.getCause() instanceof InterruptedException)\n    {\n        Thread.currentThread().interrupt();\n        // abort or retry after confirming node is up\n    }\n    else throw e;\n}","preventionTips":["Check node operation mode before issuing commits during drain/stop workflows","Never swallow InterruptedException in wrapper code around TCM commits","Order shutdown so metadata-mutating work completes before interrupting worker threads","In tests, await node readiness/initialisation before triggering transformations"],"tags":["tcm","interrupted","shutdown","cluster-metadata"],"backgroundTag":"node-shutting-down","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}