{"record":{"id":"d79013d8d406e98d","repo":"apache/cassandra","slug":"failed-marking-some-sstables-compacting-in-keyspac","errorCode":null,"errorMessage":"Failed marking some sstables compacting in keyspace %s, check server logs for more information.\n","messagePattern":"Failed marking some sstables compacting in keyspace (.+?), check server logs for more information\\.\n","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"src/java/org/apache/cassandra/tools/NodeProbe.java","lineNumber":494,"sourceCode":"                \"upgrading sstables\");\n    }\n\n    private static interface Job\n    {\n        int perform() throws IOException, ExecutionException, InterruptedException;\n    }\n\n    private void perform(PrintStream out, String ks, Job job, String jobName) throws IOException, ExecutionException, InterruptedException\n    {\n        switch (job.perform())\n        {\n            case 1:\n                out.printf(\"Aborted %s for at least one table in keyspace %s, check server logs for more information.\\n\",\n                           jobName, ks);\n                break;\n            case 2:\n                out.printf(\"Failed marking some sstables compacting in keyspace %s, check server logs for more information.\\n\", ks);\n                throw new RuntimeException(String.format(\"Failed marking some sstables compacting in keyspace %s, check server logs for more information.\\n\", ks));\n        }\n    }\n\n    public void garbageCollect(PrintStream out, String tombstoneOption, int jobs, String keyspaceName, String... tableNames) throws IOException, ExecutionException, InterruptedException\n    {\n        if (garbageCollect(tombstoneOption, jobs, keyspaceName, tableNames) != 0)\n        {\n            out.println(\"Aborted garbage collection for at least one table in keyspace \" + keyspaceName + \", check server logs for more information.\");\n            throw new RuntimeException(\"Aborted garbage collection for at least one table in keyspace \" + keyspaceName + \", check server logs for more information.\");\n        }\n    }\n\n    public void userDefinedGarbageCollect(PrintStream out, String tombstoneOption, int jobs, List<String> userDefinedTables) throws IOException, ExecutionException, InterruptedException\n    {\n        if (userDefinedGarbageCollect(tombstoneOption, jobs, userDefinedTables) != 0)\n        {\n            out.println(\"Aborted garbage collection for at least one table, check server logs for more information.\");\n            throw new RuntimeException(\"Aborted garbage collection for at least one table, check server logs for more information.\");","sourceCodeStart":476,"sourceCodeEnd":512,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/tools/NodeProbe.java#L476-L512","documentation":"After running a cleanup/scrub/verify/recompress/upgradeSSTables compaction job via JMX, NodeProbe inspects the returned status code. Status 2 means the CompactionManager could not mark some SSTables as compacting (e.g. they were being modified concurrently), so the method prints a warning and throws RuntimeException to fail nodetool.","triggerScenarios":"Calling nodetool cleanup/scrub/verify/recompressSSTables/upgradeSSTables when the server-side job returns status 2, i.e. markCompacting failed for at least one SSTable because another operation (e.g. concurrent compaction, streaming, or a parallel nodetool run) held a lock on those files.","commonSituations":"Running two compaction-related nodetool commands at once; scrub/upgradeSSTables racing with an autocompaction task; a node under heavy write load where SSTables are created/deleted while the job scans them.","solutions":["Re-run the command once concurrent compaction activity has settled","Disable autocompaction for the table (nodetool disableautocompaction) before running the job, then re-enable it","Check the server logs to identify which SSTables failed to be marked and what held them","Run only one compaction-affecting nodetool operation at a time"],"exampleFix":"// before\nnodetool scrub -- jobs 4 keyspace1\n// after\nnodetool disableautocompaction keyspace1\nnodetool scrub --jobs 4 keyspace1\nnodetool enableautocompaction keyspace1","handlingStrategy":"retry","validationCode":"// Before running, ensure no other compaction job is active:\nnodetool compactionstats | grep -E \"cleanup|scrub|verify|upgrade\" || echo OK","typeGuard":null,"tryCatchPattern":"try {\n    probe.perform(out, jobName, jobs, ks, tableNames);\n} catch (RuntimeException e) {\n    if (e.getMessage().contains(\"Failed marking\")) {\n        Thread.sleep(retryDelayMs);\n        probe.perform(out, jobName, jobs, ks, tableNames); // retry once\n    } else throw e;\n}","preventionTips":["Run one compaction-affecting nodetool command at a time","Disable autocompaction on target tables during maintenance","Schedule scrub/upgradeSSTables during low-write windows","Check server logs after any failure before retrying"],"tags":["compaction","sstables","jmx","concurrency"],"backgroundTag":"conflicting-config-options","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}