{"record":{"id":"8d63c43bf26c633c","repo":"apache/cassandra","slug":"some-operations-timed-out-details-available-at-de","errorCode":null,"errorMessage":"Some operations timed out, details available at debug level (debug.log)","messagePattern":"Some operations timed out, details available at debug level \\(debug\\.log\\)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"src/java/org/apache/cassandra/db/monitoring/MonitoringTask.java","lineNumber":175,"sourceCode":"    }\n\n    @VisibleForTesting\n    private void logOperations(long approxCurrentTimeNanos)\n    {\n        logSlowOperations(approxCurrentTimeNanos);\n        logFailedOperations(approxCurrentTimeNanos);\n\n        approxLastLogTimeNanos = approxCurrentTimeNanos;\n    }\n\n    @VisibleForTesting\n    boolean logFailedOperations(long nowNanos)\n    {\n        AggregatedOperations failedOperations = failedOperationsQueue.popOperations();\n        if (!failedOperations.isEmpty())\n        {\n            long elapsedNanos = nowNanos - approxLastLogTimeNanos;\n            noSpamLogger.warn(\"Some operations timed out, details available at debug level (debug.log)\");\n\n            if (logger.isDebugEnabled())\n                logger.debug(\"{} operations timed out in the last {} msecs:{}{}\",\n                            failedOperations.num(),\n                             NANOSECONDS.toMillis(elapsedNanos),\n                            LINE_SEPARATOR,\n                            failedOperations.getLogMessage());\n            return true;\n        }\n\n        return false;\n    }\n\n    @VisibleForTesting\n    boolean logSlowOperations(long approxCurrentTimeNanos)\n    {\n        AggregatedOperations slowOperations = slowOperationsQueue.popOperations();\n        if (!slowOperations.isEmpty())","sourceCodeStart":157,"sourceCodeEnd":193,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/db/monitoring/MonitoringTask.java#L157-L193","documentation":"MonitoringTask periodically reports timed-out monitored operations (e.g. cross-node reads/mutations that exceeded their deadlines). At WARN level it only says operations timed out; the per-operation details (name, timeout, coordinator) are emitted at DEBUG, to avoid log spam via noSpamLogger.","triggerScenarios":"Any monitored read or write operation exceeded its configured timeout within the last log interval: slow disks, network latency, GC pauses, overloaded replicas, or very low cross_node/operation timeouts in cassandra.yaml.","commonSituations":"Under-provisioned clusters under load; network partition or slow replica causing timeouts; operators seeing this generic warning but only checking the standard app log, not debug.log.","solutions":["Enable DEBUG logging for org.apache.cassandra.db.monitoring in logback.xml and inspect debug.log for the detailed list of timed-out operations","Raise cross_node_timeout / operation timeouts or investigate the slow component (disk I/O, network)","Check for GC pauses (gc.log), thread pool saturation, and dropped mutations around the same timestamps","Reduce load or add capacity if timeouts correlate with traffic spikes"],"exampleFix":null,"handlingStrategy":"retry","validationCode":"// before diagnosing, confirm clocks and timeouts are sane\nif (Math.abs(clockSkewMillis) > 500) throw new IllegalStateException(\"Clock skew across nodes\");","typeGuard":null,"tryCatchPattern":"// re-run the operation when timeouts are transient\nif (op.timedOut()) retry(op, backoff);","preventionTips":["Enable DEBUG on org.apache.cassandra.db.monitoring to get details in debug.log","Right-size cross_node timeouts to p99 latencies","Monitor GC pauses and disk I/O alongside this warning","Load-test before traffic peaks"],"tags":["timeout","monitoring","performance"],"backgroundTag":"request-timeout","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}