conductor-oss/conductor · error · TransientException

Failed to remove task: %s

Error message

Failed to remove task: %s

What it means

removeTask wraps DriverException in a TransientException naming the task id. The removal executes a batch (task delete + lookup delete) and optionally removes the task from its concurrency limit; any driver-level failure is retried by the framework RetryTemplate.

Source

Thrown at cassandra-persistence/src/main/java/com/netflix/conductor/cassandra/dao/CassandraExecutionDAO.java:803

                            UUID.fromString(task.getWorkflowInstanceId()),
                            DEFAULT_SHARD_ID,
                            task.getTaskId()));
            batchStatement.add(
                    updateTotalTasksStatement.bind(
                            totalTasks - 1,
                            UUID.fromString(task.getWorkflowInstanceId()),
                            DEFAULT_SHARD_ID));
            ResultSet resultSet = session.execute(batchStatement);
            if (task.getTaskDefinition().isPresent()
                    && task.getTaskDefinition().get().concurrencyLimit() > 0) {
                removeTaskFromLimit(task);
            }
            return resultSet.wasApplied();
        } catch (DriverException e) {
            Monitors.error(CLASS_NAME, "removeTask");
            String errorMsg = String.format("Failed to remove task: %s", task.getTaskId());
            LOGGER.error(errorMsg, e);
            throw new TransientException(errorMsg);
        }
    }

    protected void removeTaskLookup(TaskModel task) {
        try {
            recordCassandraDaoRequests(
                    "removeTaskLookup", task.getTaskType(), task.getWorkflowType());
            if (task.getTaskDefinition().isPresent()
                    && task.getTaskDefinition().get().concurrencyLimit() > 0) {
                removeTaskFromLimit(task);
            }
            session.execute(deleteTaskLookupStatement.bind(UUID.fromString(task.getTaskId())));
        } catch (DriverException e) {
            Monitors.error(CLASS_NAME, "removeTaskLookup");
            String errorMsg = String.format("Failed to remove task lookup: %s", task.getTaskId());
            LOGGER.error(errorMsg, e);
            throw new TransientException(errorMsg);
        }

View on GitHub (pinned to cf7c3e4a8a)

Solutions

1. Confirm Cassandra health and that the batch write consistency is satisfiable.
2. Tune DataStax driver batch/timeout and connection pool settings.
3. Use the built-in retry; if exhausted, examine the logged DriverException cause.
4. Check for oversized batches if a workflow has very many tasks.

RetryTemplate retry = RetryTemplate.builder().retryOn(TransientException.class).maxAttempts(3).noBackoff().build();
return retry.execute(ctx -> removeTask(task));

When it happens

Trigger: The batched task removal fails at the driver level: batch write timeout, node unavailable, consistency not met, or a nested removeTaskFromLimit failure.

Common situations: Cluster stress during workflow sweep/scheduling, driver pool exhaustion, RF/consistency mismatch under node loss.

Related errors

• Failed to remove task lookup: %s
• Failed to lookup workflowId from taskId: %s
• Failed to get workflow: %s
• Failed to add event execution for event: %s, handler: %s
• Failed to update event execution for event: %s, handler: %s

AI-assisted analysis of conductor-oss/conductor@cf7c3e4a8a (2026-08-14). Data as JSON: /api/errors/0b3c6859dc95e86d. Report an issue: GitHub.