apache/cassandra · warning · IllegalStateException

Training already in progress for table <keyspaceName>.<table

Error message

Training already in progress for table <keyspaceName>.<tableName>

What it means

CompressionDictionaryScheduler serializes SSTable-based training per table using an atomic trainingInProgress flag. If scheduleSSTableBasedTraining() is invoked while a training run is already in flight for the same table, the CAS fails, the referenced SSTable view is released, and IllegalStateException is thrown. Only one training at a time per table is permitted.

Source

Thrown at src/java/org/apache/cassandra/db/compression/CompressionDictionaryScheduler.java:100

        this.scheduledRefreshTask = ScheduledExecutors.scheduledTasks.scheduleWithFixedDelay(
        this::refreshDictionaryFromSystemTable,
        DatabaseDescriptor.getCompressionDictionaryRefreshInitialDelaySeconds(),
        DatabaseDescriptor.getCompressionDictionaryRefreshIntervalSeconds(),
        TimeUnit.SECONDS
        );
    }

    @Override
    public void scheduleSSTableBasedTraining(ColumnFamilyStore.RefViewFragment refViewFragment,
                                             CompressionParams compressionParams,
                                             CompressionDictionaryTrainingConfig config,
                                             Consumer<CompressionDictionary> listener,
                                             boolean force)
    {
        if (!trainingInProgress.compareAndSet(false, true))
        {
            refViewFragment.close();
            throw new IllegalStateException("Training already in progress for table " + keyspaceName + '.' + tableName);
        }

        ICompressionDictionaryTrainer trainer;

        try
        {
            trainer = ICompressionDictionaryTrainer.create(keyspaceName, tableName, compressionParams);
            trainer.setDictionaryTrainedListener(listener);
        }
        catch (Throwable t)
        {
            trainingInProgress.set(false);
            refViewFragment.close();
            throw t;
        }

        if (trainer.start(config))
        {

View on GitHub (pinned to 88fd0f6a0e)

Solutions

  1. Wait for the in-progress training to complete before requesting another; check the training status via JMX first.
  2. Catch IllegalStateException and treat it as 'training already running' rather than an error in automation.
  3. Serialize training requests through a single job/queue per table.
  4. If a previous training appears stuck, investigate/restart the node to clear the flag rather than forcing concurrent runs.

Example fix

// before
try { manager.train(true, params); }
catch (Exception e) { throw e; }
// after
try { manager.train(true, params); }
catch (IllegalStateException e) {
    if (e.getMessage().contains("Training already in progress")) return; // already running
    throw e;
}
Defensive patterns

Strategy: try-catch

Validate before calling

if (!manager.isTrainingInProgress()) manager.train(force, params);

Try / catch

try { manager.train(force, params); } catch (IllegalStateException e) { if (e.getMessage().contains("already in progress")) { /* skip or retry later */ } else throw e; }

Prevention

When it happens

Trigger: Calling scheduleSSTableBasedTraining() (via train() or scheduled compaction-triggered training) while another training session for the same keyspace.table has not yet completed.

Common situations: Two operators or scripts triggering training concurrently; a scheduled periodic trainer overlapping a manual training; a slow/stuck previous training run still holding the flag when a new request arrives.

Understand the failure class

Background: "Invalid state transition" errors: "status must be X, actually Y", "already rejected/charging/uninstalled", "cannot ... while running" — what they mean when a library rejects your call — this error's family across 31 libraries.

Related errors


AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/39129c8028d7029d. Report an issue: GitHub.