apache/cassandra · error · java.lang.RuntimeException

Error occurred during scrubbing

Error message

Error occurred during scrubbing

What it means

Nodetool Scrub re-throws IllegalArgumentException unchanged (bad user arguments) but wraps every other exception from the scrub operation in a RuntimeException 'Error occurred during scrubbing'. The scrub may have already begun on the node, so check how far it progressed before retrying.

Source

Thrown at src/java/org/apache/cassandra/tools/nodetool/Scrub.java:89

    public void execute(NodeProbe probe)
    {
        args = concatArgs(keyspace, tables);
        List<String> keyspaces = CommandUtils.parseOptionalKeyspace(args, probe);
        String[] tableNames = CommandUtils.parseOptionalTables(args);

        for (String keyspace : keyspaces)
        {
            try
            {
                probe.scrub(probe.output().out, disableSnapshot, skipCorrupted, !noValidation, reinsertOverflowedTTL, jobs, keyspace, tableNames);
            }
            catch (IllegalArgumentException e)
            {
                throw e;
            }
            catch (Exception e)
            {
                throw new RuntimeException("Error occurred during scrubbing", e);
            }
        }
    }
}

View on GitHub (pinned to 88fd0f6a0e)

Solutions

  1. Check node logs and the wrapped cause for the failing SSTable
  2. Free disk space — scrub needs room to rewrite SSTables
  3. Re-run scrub with --no-skip or recover segments; use `nodetool verifysstable`/sstablemetadata to isolate corrupt files
  4. Restore corrupt SSTables from backup if scrub cannot fix them

Example fix

// before
nodetool scrub myks (with <5% free disk)
// after
df -h  # ensure ample free space, then
nodetool scrub myks
Defensive patterns

Strategy: validation

Validate before calling

df -h "$DATA_DIR" | awk 'NR==2 && $5+0 > 90 {print "insufficient disk for scrub"; exit 1}'

Try / catch

try { probe.scrub(out, options, ks, tables); } catch (IllegalArgumentException e) { throw e; /* bad args: fix invocation */ } catch (RuntimeException e) { log.error("scrub failed: {}", e.getCause(), e); /* inspect corrupt sstables before retry */ throw e; }

Prevention

When it happens

Trigger: `nodetool scrub` when the node-side scrub fails mid-operation: corrupt SSTables that cannot be read/moved, IO errors, invalid combination of scrub options rejected remotely (non-IAE).

Common situations: Scrubbing genuinely corrupted SSTables after a crash; disk-space exhaustion during scrub (scrub writes new SSTables); running scrub concurrently with other compactions.

Understand the failure class

Background: "API request failed": what wrapped HTTP errors from external APIs mean and how to find the real cause — this error's family across 29 libraries.

Related errors


AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/2e9002cb492eb44f. Report an issue: GitHub.