apache/cassandra · warning

Failed to remove hard link files

Error message

Failed to remove hard link files

What it means

ComponentContext manages hard links created for streamed sstable components and deletes them on close. deleteWithConfirm accumulates any failures; if the accumulated exception is non-null, close logs 'Failed to remove hard link files' as a warning. The links are typically already superseded; this signals leftover files consuming disk until manually removed.

Source

Thrown at src/java/org/apache/cassandra/db/streaming/ComponentContext.java:96

        @SuppressWarnings("resource") // file channel will be closed by Caller
        FileChannel channel = toTransfer.newReadChannel();

        assert size == channel.size() : String.format("Entire sstable streaming expects %s file size to be %s but got %s.",
                                                      component, size, channel.size());
        return channel;
    }

    @Override
    public void close()
    {
        Throwable accumulate = null;
        for (File file : hardLinks.values())
            accumulate = FileUtils.deleteWithConfirm(file, accumulate);

        hardLinks.clear();

        if (accumulate != null)
            logger.warn("Failed to remove hard link files", accumulate);
    }
}

View on GitHub (pinned to 88fd0f6a0e)

Solutions

1. Manually delete leftover hard-link files from the streamed sstable directories after confirming no process uses them
2. Check directory permissions for the Cassandra user on the data directories
3. Look at the accumulated cause in the log to identify which file failed and why
4. If on NFS/EBS-backed storage, verify the filesystem supports unlink semantics Cassandra expects

// ensure Cassandra user owns data dirs and no stale processes hold files
// ls -l on hardlink dirs; lsof +D <data_dir> before manual cleanup

try (ComponentContext ctx = new ComponentContext(...)) {
    // use component
} // close() logs warning on delete failure; clean leftover files manually afterward

When it happens

Trigger: close() is called (stream writer aborted or completed) and one or more hard-linked files cannot be deleted — usually because another process holds them open, or a permission/IO error prevents removal.

Common situations: Files still open by a concurrent reader at close time; filesystem permission problems; NFS/EFS quirkiness where unlink of an open file behaves unexpectedly; disk errors.

Related errors

• Unable to hardlink from %s to %s
• Unable to delete storage-attached index component file {} du
• Exception thrown when cleaning up files to delete on exit, c
• Failed importing SSTables
• Unable to resolve canonical path for

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/5e745100d4f00a0a. Report an issue: GitHub.