apache/cassandra · error · RuntimeException

Cannot remove temporary or obsoleted files for %s.%s due to

Error message

Cannot remove temporary or obsoleted files for %s.%s due to a problem with transaction log files.

What it means

SSTableLevelResetter, before resetting leveled-compaction levels offline, cleans up leftover temporary/obsoleted SSTables via LifecycleTransaction.removeUnfinishedLeftovers. If that cleanup reports failure (inconsistent or unreadable transaction log files), it throws a RuntimeException naming the keyspace and table.

Source

Thrown at src/java/org/apache/cassandra/tools/SSTableLevelResetter.java:83

        // TODO several daemon threads will run from here.
        // So we have to explicitly call System.exit.
        try
        {
            String keyspaceName = args[1];
            String columnfamily = args[2];
            // validate columnfamily
            if (Schema.instance.getTableMetadata(keyspaceName, columnfamily) == null)
            {
                System.err.println("ColumnFamily not found: " + keyspaceName + "/" + columnfamily);
                System.exit(1);
            }

            // remove any leftovers in the transaction log
            Keyspace keyspace = Keyspace.openWithoutSSTables(keyspaceName);
            ColumnFamilyStore cfs = keyspace.getColumnFamilyStore(columnfamily);
            if (!LifecycleTransaction.removeUnfinishedLeftovers(cfs))
            {
                throw new RuntimeException(String.format("Cannot remove temporary or obsoleted files for %s.%s " +
                                                         "due to a problem with transaction log files.",
                                                         keyspace, columnfamily));
            }

            Directories.SSTableLister lister = cfs.getDirectories().sstableLister(Directories.OnTxnErr.THROW).skipTemporary(true);
            boolean foundSSTable = false;
            for (Map.Entry<Descriptor, Set<Component>> sstable : lister.list().entrySet())
            {
                if (sstable.getValue().contains(Components.STATS))
                {
                    foundSSTable = true;
                    Descriptor descriptor = sstable.getKey();
                    StatsMetadata metadata = StatsComponent.load(descriptor).statsMetadata();
                    if (metadata.sstableLevel > 0)
                    {
                        out.println("Changing level from " + metadata.sstableLevel + " to 0 on " + descriptor.fileFor(Components.DATA));
                        descriptor.getMetadataSerializer().mutateLevel(descriptor, 0);
                    }

View on GitHub (pinned to 88fd0f6a0e)

Solutions

  1. Inspect the table directory's *txn.log files and the SSTables they reference; restore missing SSTables from a replica/backup
  2. Remove stale/orphaned transaction log files only after confirming the referenced SSTables' status
  3. Re-run sstablelevelreset after the transaction logs are consistent
  4. Take a filesystem backup before manual txn-log surgery
Defensive patterns

Strategy: validation

Validate before calling

// before running, confirm txn logs and referenced SSTables are present
for (File txn : new File(tableDir).listFiles((d, n) -> n.endsWith(".txn.log")))
    if (!txn.canRead()) throw new IllegalStateException("Unreadable txn log: " + txn);

Try / catch

try { SSTableLevelResetter.main(args); }
catch (RuntimeException e) { LOG.error("leftover cleanup failed: " + e.getMessage(), e); }

Prevention

When it happens

Trigger: Running `sstablelevelreset` on a table whose txn (transaction log) files are corrupted, partially written, or reference SSTables that are missing, so removeUnfinishedLeftovers(cfs) returns false.

Common situations: Crashed compaction/flush left inconsistent txn logs; someone manually deleted SSTable files referenced by transaction logs; running the tool on a data directory copied mid-operation; wrong keyspace/table argument pointing at unexpected leftovers.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors


AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/10af3cc3658bd7e9. Report an issue: GitHub.