apache/cassandra · error · RuntimeException

Failed marking some sstables compacting in keyspace %s, chec

Error message

Failed marking some sstables compacting in keyspace %s, check server logs for more information.

What it means

After running a cleanup/scrub/verify/recompress/upgradeSSTables compaction job via JMX, NodeProbe inspects the returned status code. Status 2 means the CompactionManager could not mark some SSTables as compacting (e.g. they were being modified concurrently), so the method prints a warning and throws RuntimeException to fail nodetool.

Source

Thrown at src/java/org/apache/cassandra/tools/NodeProbe.java:494

                "upgrading sstables");
    }

    private static interface Job
    {
        int perform() throws IOException, ExecutionException, InterruptedException;
    }

    private void perform(PrintStream out, String ks, Job job, String jobName) throws IOException, ExecutionException, InterruptedException
    {
        switch (job.perform())
        {
            case 1:
                out.printf("Aborted %s for at least one table in keyspace %s, check server logs for more information.\n",
                           jobName, ks);
                break;
            case 2:
                out.printf("Failed marking some sstables compacting in keyspace %s, check server logs for more information.\n", ks);
                throw new RuntimeException(String.format("Failed marking some sstables compacting in keyspace %s, check server logs for more information.\n", ks));
        }
    }

    public void garbageCollect(PrintStream out, String tombstoneOption, int jobs, String keyspaceName, String... tableNames) throws IOException, ExecutionException, InterruptedException
    {
        if (garbageCollect(tombstoneOption, jobs, keyspaceName, tableNames) != 0)
        {
            out.println("Aborted garbage collection for at least one table in keyspace " + keyspaceName + ", check server logs for more information.");
            throw new RuntimeException("Aborted garbage collection for at least one table in keyspace " + keyspaceName + ", check server logs for more information.");
        }
    }

    public void userDefinedGarbageCollect(PrintStream out, String tombstoneOption, int jobs, List<String> userDefinedTables) throws IOException, ExecutionException, InterruptedException
    {
        if (userDefinedGarbageCollect(tombstoneOption, jobs, userDefinedTables) != 0)
        {
            out.println("Aborted garbage collection for at least one table, check server logs for more information.");
            throw new RuntimeException("Aborted garbage collection for at least one table, check server logs for more information.");

View on GitHub (pinned to 88fd0f6a0e)

Solutions

1. Re-run the command once concurrent compaction activity has settled
2. Disable autocompaction for the table (nodetool disableautocompaction) before running the job, then re-enable it
3. Check the server logs to identify which SSTables failed to be marked and what held them
4. Run only one compaction-affecting nodetool operation at a time

Example fix

// before
nodetool scrub -- jobs 4 keyspace1
// after
nodetool disableautocompaction keyspace1
nodetool scrub --jobs 4 keyspace1
nodetool enableautocompaction keyspace1

// Before running, ensure no other compaction job is active:
nodetool compactionstats | grep -E "cleanup|scrub|verify|upgrade" || echo OK

try {
    probe.perform(out, jobName, jobs, ks, tableNames);
} catch (RuntimeException e) {
    if (e.getMessage().contains("Failed marking")) {
        Thread.sleep(retryDelayMs);
        probe.perform(out, jobName, jobs, ks, tableNames); // retry once
    } else throw e;
}

When it happens

Trigger: Calling nodetool cleanup/scrub/verify/recompressSSTables/upgradeSSTables when the server-side job returns status 2, i.e. markCompacting failed for at least one SSTable because another operation (e.g. concurrent compaction, streaming, or a parallel nodetool run) held a lock on those files.

Common situations: Running two compaction-related nodetool commands at once; scrub/upgradeSSTables racing with an autocompaction task; a node under heavy write load where SSTables are created/deleted while the job scans them.

Understand the failure class

Background: Conflicting config options: "cannot be used together" — configuration validation errors across open-source libraries — this error's family across 162 libraries.

Related errors

• Could not set new local compaction strategy: <cause message>
• The min_compaction_threshold cannot be larger than the max_c
• Disabling compaction by setting min_compaction_threshold or
• Prepare phase for incremental repair session %s has failed b
• Number of concurrent compactors should be greater than 0.

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/d79013d8d406e98d. Report an issue: GitHub.