{"record":{"id":"ba86b62d71fee304","repo":"conductor-oss/conductor","slug":"failed-to-check-rate-limit-for-task-s-in-taskdef","errorCode":null,"errorMessage":"Failed to check rate limit for task: %s in taskDef: %s","messagePattern":"Failed to check rate limit for task: (.+?) in taskDef: (.+?)","errorType":"exception","errorClass":"TransientException","httpStatus":null,"severity":"error","filePath":"cassandra-persistence/src/main/java/com/netflix/conductor/cassandra/dao/CassandraRateLimitingDAO.java","lineNumber":134,"sourceCode":"                return false;\n            } else {\n                LOGGER.info(\n                        \"TaskId: {} with TaskDefinition of: {} has rateLimitPerFrequency: {} and rateLimitFrequencyInSeconds: {} is out of bounds of rate limit with current count {}\",\n                        task.getTaskId(),\n                        task.getTaskDefName(),\n                        rateLimitPerFrequency,\n                        rateLimitFrequencyInSeconds,\n                        currentBucketCount);\n                return true;\n            }\n        } catch (DriverException e) {\n            Monitors.error(CLASS_NAME, \"exceedsRateLimitPerFrequency\");\n            String errorMsg =\n                    String.format(\n                            \"Failed to check rate limit for task: %s in taskDef: %s\",\n                            task.getTaskId(), task.getTaskDefName());\n            LOGGER.error(errorMsg, e);\n            throw new TransientException(errorMsg, e);\n        }\n    }\n}\n","sourceCodeStart":116,"sourceCodeEnd":138,"githubUrl":"https://github.com/conductor-oss/conductor/blob/cf7c3e4a8adfb158be778ab1ec525323c363cd3a/cassandra-persistence/src/main/java/com/netflix/conductor/cassandra/dao/CassandraRateLimitingDAO.java#L116-L138","documentation":"Thrown when a DriverException occurs inside exceedsRateLimitPerFrequency while counting the current rate-limit bucket (SELECT count) or recording a new execution (INSERT timeuuid). Wrapped as TransientException - the rate-limit check could not complete due to an infrastructure failure. Note the rate limit itself is approximate: count and insert are separate statements, so concurrent pollers can overshoot.","triggerScenarios":"session.execute(selectRateLimitCountStatement.bind(...)) or session.execute(insertRateLimitBucketStatement.bind(...)) raises a DriverException during the per-frequency rate-limit evaluation for a task.","commonSituations":"Cassandra degradation during high poll throughput; task_rate_limit table missing (schema not migrated); read consistency below quorum on a multi-node cluster returning a stale window; connection pool exhaustion.","solutions":["Retry the rate-limit check with backoff - it is a TransientException.","Verify the task_rate_limit table exists (run the schema migration).","Ensure conductor.cassandra.readConsistencyLevel is at least quorum on multi-node clusters to avoid stale window counts.","Check the wrapped DriverException for timeout vs unavailable."],"exampleFix":"// before: single check, surfaces TransientException\nboolean limited = rateLimitingDAO.exceedsRateLimitPerFrequency(task, taskDef);\n\n// after: retry transient failures\nboolean limited = RetryUtils.retryOn(TransientException.class, 3,\n        Duration.ofMillis(100),\n        () -> rateLimitingDAO.exceedsRateLimitPerFrequency(task, taskDef));","handlingStrategy":"retry","validationCode":"// Pre-flight: confirm session before rate-limit check\nif (cassandraSession.isClosed()) {\n    throw new IllegalStateException(\"Cassandra session is closed; cannot check rate limit\");\n}","typeGuard":null,"tryCatchPattern":"// Retry transient rate-limit checks; on persistent failure, fail safe per policy\ntry {\n    return rateLimitingDAO.exceedsRateLimitPerFrequency(task, taskDef);\n} catch (TransientException e) {\n    return backoffAndRetry(\n            () -> rateLimitingDAO.exceedsRateLimitPerFrequency(task, taskDef), 3);\n}","preventionTips":["Retry exceedsRateLimitPerFrequency on TransientException with backoff.","Keep read consistency at quorum+ on multi-node clusters for accurate window counts.","Ensure the task_rate_limit table exists (run the schema migration).","Decide a fail-safe vs fail-closed policy if the check stays unavailable."],"tags":["cassandra","rate-limiting","transient","network","driver"],"backgroundTag":null,"analyzedSha":"cf7c3e4a8adfb158be778ab1ec525323c363cd3a","analyzedAt":"2026-08-14T03:33:19.897Z","schemaVersion":2},"datasetVersion":"2026-08-14T05:17:29.042Z"}