conductor-oss/conductor · error · TransientException

Failed to get in progress limit - %s:%s in workflow :%s

Error message

Failed to get in progress limit - %s:%s in workflow :%s

What it means

Thrown by CassandraExecutionDAO.exceedsLimit when reading the in-progress count for a task's rate limit fails. DriverException during the limit lookup is wrapped in TransientException (note: cause is not attached). Reports taskDefName, taskId, and workflowInstanceId.

Source

Thrown at cassandra-persistence/src/main/java/com/netflix/conductor/cassandra/dao/CassandraExecutionDAO.java:333

            if (!taskIds.contains(task.getTaskId()) && current >= limit) {
                LOGGER.info(
                        "Task execution count limited. task - {}:{}, limit: {}, current: {}",
                        task.getTaskId(),
                        task.getTaskDefName(),
                        limit,
                        current);
                Monitors.recordTaskConcurrentExecutionLimited(task.getTaskDefName(), limit);
                return true;
            }
        } catch (DriverException e) {
            Monitors.error(CLASS_NAME, "exceedsLimit");
            String errorMsg =
                    String.format(
                            "Failed to get in progress limit - %s:%s in workflow :%s",
                            task.getTaskDefName(), task.getTaskId(), task.getWorkflowInstanceId());
            LOGGER.error(errorMsg, e);
            throw new TransientException(errorMsg);
        }
        return false;
    }

    @Override
    public boolean removeTask(String taskId) {
        TaskModel task = getTask(taskId);
        if (task == null) {
            LOGGER.warn("No such task found by id {}", taskId);
            return false;
        }
        return removeTask(task);
    }

    @Override
    public TaskModel getTask(String taskId) {
        try {
            String workflowId = lookupWorkflowIdFromTaskId(taskId);

View on GitHub (pinned to cf7c3e4a8a)

Solutions

  1. Confirm Cassandra is reachable and the task_def_rate_limit table exists (init() ran).
  2. Retry the exceedsLimit check — TransientException is retryable.
  3. If the failure is persistent, temporarily verify the rate-limit config and table schema.
  4. Attach the cause exception in the throw to improve diagnosis (it is currently omitted).

Example fix

// before
throw new TransientException(errorMsg);

// after (preserve the cause for diagnosis)
throw new TransientException(errorMsg, e);
Defensive patterns

Strategy: retry

Validate before calling

// Only check limit when a TaskDef with a limit is present
if (task.getTaskDefinition().filter(td -> td.concurrencyLimit() > 0).isEmpty()) return false;

Try / catch

try {
    return dao.exceedsLimit(task);
} catch (TransientException e) {
    log.warn("Transient exceedsLimit check for {}; retrying", task.getTaskId());
    throw e;

Prevention

When it happens

Trigger: Calling exceedsLimit(TaskModel) for a task whose TaskDef has a concurrentExecutionLimit set, when the SELECT against the task rate-limit table fails (Cassandra unavailable, timeout, or schema missing).

Common situations: Cassandra connectivity loss. task_def_rate_limit table not initialized. Coordinator timeout under contention when many tasks check the limit concurrently.

Related errors


AI-assisted analysis of conductor-oss/conductor@cf7c3e4a8a (2026-08-14). Data as JSON: /api/errors/2312096f85feced8. Report an issue: GitHub.