apache/cassandra · warning · RejectedExecutionException

Scheduler has shut down.

Error message

Scheduler has shut down.

What it means

AccordScheduler.now(Runnable) submits a task to run immediately on its scheduled executor. If the executor has already been shut down (node shutdown/decommission of the Accord service), submit() would be rejected, so the method explicitly throws RejectedExecutionException('Scheduler has shut down.'). This indicates work was scheduled during or after Accord teardown.

Solutions

1. Check scheduler.isTerminated()/isShutdown() (or catch RejectedExecutionException) before/when scheduling and drop the task during shutdown
2. Ensure node shutdown quiesces transactional/mutation-stage work before stopping the Accord scheduler
3. In tests, close clusters only after all in-flight async work completes
4. Re-submit the task if the scheduler is expected to be restarted, or route the work to a valid executor

Example fix

// before
scheduler.now(task);
// after
try
{
    scheduler.now(task);
}
catch (RejectedExecutionException e)
{
    logger.info("Dropping task {}; scheduler has shut down", task, e);
}

if (scheduler.isTerminated() || scheduler.isShutdown()) { logger.info("Scheduler shut down; dropping task"); return; }

boolean canSchedule(AccordScheduler s) { return !s.isShutdown() && !s.isTerminated(); }

try { scheduler.now(task); } catch (RejectedExecutionException e) { logger.debug("Task dropped after scheduler shutdown", e); }

When it happens

Trigger: Calling AccordScheduler.now(task) after Scheduler shutdown began — e.g. a mutation-stage verb or timer callback firing while the node is shutting down, decommissioning, or Accord is being stopped.

Common situations: In-flight transactional work racing node shutdown/decommission; tests that tear down the cluster while timers still fire; restart/stop scripts executed while Accord tasks are pending; error handling paths that schedule retries post-shutdown.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors

• HintsService has already been shut down
• HintsService is shut down and can't accept new hints
• HintsService is shut down and cannot be restarted
• Not executing task on GossipStage - it has shut down
• ScheduledThreadPoolExecutor has shut down.

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/605ad85ececfee5b. Report an issue: GitHub.

Appendix: source

Thrown at src/java/org/apache/cassandra/service/accord/api/AccordScheduler.java:85

    public Scheduled once(Runnable run, long delay, TimeUnit units)
    {
        ScheduledFuture<?> future = scheduledExecutor.schedule(run, delay, units);
        return new ScheduledFutureWrapper(future);
    }

    @Override
    public Scheduled selfRecurring(Runnable run, long delay, TimeUnit units)
    {
        ScheduledFuture<?> future = scheduledExecutor.scheduleSelfRecurring(run, delay, units);
        return new ScheduledFutureWrapper(future);
    }

    @Override
    public void now(Runnable task)
    {
        // called from the mutation stage configured by the verb
        if (scheduledExecutor.isShutdown())
            throw new RejectedExecutionException("Scheduler has shut down.");
        scheduledExecutor.submit(task);
    }

    @Override
    public boolean isTerminated()
    {
        return scheduledExecutor.isTerminated();
    }

    @Override
    public void shutdown()
    {
        for (Runnable c : shutdownNow())
        {
            if (c instanceof java.util.concurrent.Future<?>)
                ((java.util.concurrent.Future<?>) c).cancel(false);
        }
    }

View on GitHub (pinned to 88fd0f6a0e)