{"record":{"id":"346ca0c0ec4f5d91","repo":"apache/cassandra","slug":"write-timeout","errorCode":"WRITE_TIMEOUT","errorMessage":"Operation timed out - received 0/%d responses.","messagePattern":"Operation timed out - received 0/(.+?) responses\\.","errorType":"exception","errorClass":"WriteTimeoutException","httpStatus":null,"severity":"warning","filePath":"src/java/org/apache/cassandra/service/paxos/PaxosState.java","lineNumber":429,"sourceCode":"\n    // don't increment the total count, as we are only using this for locking purposes when coordinating\n    @VisibleForTesting\n    public static PaxosOperationLock lock(DecoratedKey partitionKey, TableMetadata metadata, long deadline, ConsistencyLevel consistencyForConsensus, boolean isWrite) throws RequestTimeoutException\n    {\n        if (DISABLE_COORDINATOR_LOCKING)\n            return PaxosOperationLock.noOp();\n\n        PaxosState lock = ACTIVE.compute(new Key(partitionKey, metadata), (key, cur) -> {\n            if (cur == null)\n                cur = new PaxosState(key, RECENT.remove(key));\n            ++cur.active;\n            return cur;\n        });\n\n        try\n        {\n            if (!lock.lock(deadline))\n                throw throwTimeout(metadata, consistencyForConsensus, isWrite);\n            return lock;\n        }\n        catch (Throwable t)\n        {\n            lock.close();\n            throw t;\n        }\n    }\n    \n    private static RequestTimeoutException throwTimeout(TableMetadata metadata, ConsistencyLevel consistencyForConsensus, boolean isWrite)\n    {\n        int blockFor = consistencyForConsensus.blockFor(Keyspace.open(metadata.keyspace).getReplicationStrategy());\n        throw isWrite\n                ? new WriteTimeoutException(WriteType.CAS, consistencyForConsensus, 0, blockFor)\n                : new ReadTimeoutException(consistencyForConsensus, 0, blockFor, false);\n    }\n\n    private PaxosState maybeLoad()","sourceCodeStart":411,"sourceCodeEnd":447,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/service/paxos/PaxosState.java#L411-L447","documentation":"PaxosState.lock() attempts to acquire the per-key Paxos lock within the caller's deadline. If the lock cannot be acquired before the deadline expires, a TimeoutException (code WRITE_TIMEOUT) is thrown with a message reporting how many responses were received out of the required block-for (here 0). This surfaces that the coordinator could not even begin the consensus round in time.","triggerScenarios":"Calling PaxosState.lock() under write request timeout pressure: concurrent Paxos transactions contending on the same partition key, or a node/network too slow to grant the lock before the deadline (e.g. a query with low USING TIMEOUT or an overloaded cluster).","commonSituations":"Hot-partition contention where many LWT (lightweight transaction) writes race on the same row; severely overloaded or GC-pausing replicas; client-side timeouts set too low for CAS operations; Repair/Paxos ballot storms.","solutions":["Retry the operation with backoff — Paxos locks are transient contention, not permanent failure","Reduce concurrent writers to the same partition key or shard the hot key","Increase the client write timeout / USING TIMEOUT for LWT-heavy workloads","Check replica health (GC pauses, dropped messages, network latency) with nodetool tpstats/nodetool netstats","Consider reducing CAS usage on hot keys or using a different data model"],"exampleFix":"// before\nsession.execute(\"UPDATE t SET v = 1 WHERE k = ? IF v = 0\", key); // WRITE_TIMEOUT under contention\n// after\nfor (int attempt = 0; attempt < 3; attempt++) {\n    try {\n        session.execute(\"UPDATE t SET v = 1 WHERE k = ? IF v = 0\", key);\n        break;\n    } catch (WriteTimeoutException e) {\n        Thread.sleep(50L * (attempt + 1));\n    }\n}","handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"try {\n    return paxosOperation();\n} catch (WriteTimeoutException e) {\n    if (e.blockFor() == 0 || e.receivedAcknowledgments() == 0)\n        backoffAndRetry(e, attempt); // lock contention, safe to retry\n    else\n        throw e;\n}","preventionTips":["Avoid hot keys with high CAS concurrency; shard or queue writes per key","Set generous timeouts for LWT statements (they need multiple round trips)","Monitor replica GC pauses and dropped mutations that delay lock grants","Implement bounded retry with jittered backoff for Paxos WRITE_TIMEOUT with 0 responses"],"tags":["timeout","paxos","lwt","contention"],"backgroundTag":"request-timeout","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}