apache/cassandra · critical · RuntimeException

Not enough disk space for stream %s), summaries=%s

Error message

Not enough disk space for stream %s), summaries=%s

What it means

Before accepting an incoming stream prepare, the node estimates the disk space the requested stream summaries would consume, accounting for ongoing compactions. If projected free space is insufficient it throws this RuntimeException, refusing the stream to avoid filling the disk.

Source

Thrown at src/java/org/apache/cassandra/streaming/StreamSession.java:917

     * TODO: Consider revising this to returning a boolean and allowing callers upstream to handle that.
     */
    private void checkAvailableDiskSpaceAndCompactions(Collection<StreamSummary> summaries)
    {
        if (DatabaseDescriptor.getSkipStreamDiskSpaceCheck())
            return;

        boolean hasAvailableSpace = true;

        try
        {
            hasAvailableSpace = checkAvailableDiskSpaceAndCompactions(summaries, planId(), peer.getHostAddress(true), pendingRepair != null);
        }
        catch (Exception e)
        {
            logger.error("[Stream #{}] Could not check available disk space and compactions for {}, summaries = {}", planId(), this, summaries, e);
        }
        if (!hasAvailableSpace)
            throw new RuntimeException(String.format("Not enough disk space for stream %s), summaries=%s", this, summaries));
    }

    /**
     * Makes sure that we expect to have enough disk space available for the new streams, taking into consideration
     * the ongoing compactions and streams.
     */
    @VisibleForTesting
    public static boolean checkAvailableDiskSpaceAndCompactions(Collection<StreamSummary> summaries,
                                                                @Nullable TimeUUID planId,
                                                                @Nullable String remoteAddress,
                                                                boolean isForIncremental)
    {
        Map<TableId, Long> perTableIdIncomingBytes = new HashMap<>();
        Map<TableId, Integer> perTableIdIncomingFiles = new HashMap<>();
        long newStreamTotal = 0;
        for (StreamSummary summary : summaries)
        {
            perTableIdIncomingFiles.merge(summary.tableId, summary.files, Integer::sum);

View on GitHub (pinned to 88fd0f6a0e)

Solutions

1. Free disk space on the streaming target node (run cleanup, nodetool compaction, archive/delete old snapshots).
2. Reduce concurrency: run nodetool stop followed by smaller batches of repair/rebuild so less data streams at once.
3. Lower concurrent_compactors / compaction throughput temporarily to reduce reserved space, or wait for compactions to finish.
4. Expand disk capacity or rebalance tokens so less data lands on the full node.
5. Monitor with nodetool diskusage / metrics (disk utilization, stream totals) before starting large streams.

Example fix

// before
// node ran out of space mid-stream
// after
// check before streaming:
long usable = diskUsableBytes(targetDir);
long needed = summaries.stream().mapToLong(s -> s.estimateSize()).sum();
if (usable < needed * 2) {
    throw new IllegalStateException("Refusing stream: insufficient disk space");
}

long usable = fileStore.getUsableSpace();
long needed = summaries.stream().mapToLong(StreamSummary::getTotalSize).sum();
if (usable < needed * 2) { /* free space or refuse */ }

try { session.prepare(msg); } catch (RuntimeException e) { if (e.getMessage().contains("Not enough disk space")) { cleanupDisk(); retryLater(); } else { throw e; } }

When it happens

Trigger: prepareAsync or prepareSynAck calling checkAvailableDiskSpaceAndCompactions and finding !hasAvailableSpace — the requested summaries' total size exceeds available disk space given current compaction reserves.

Common situations: Streaming a large amount of data (bootstrap, repair, decommission) onto a node whose disks are nearly full; many concurrent compactions reserving space; under-provisioned nodes in busy clusters.

Related errors

• Insufficient disk space to store %s
• Only {} free across all data volumes. Consider adding more c
• Invalid value of stream_throughput_outbound:
• Invalid value of inter_dc_stream_throughput_outbound:
• stream_throughput_outbound: is too large; it should be less

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/44322268d0e1a1d9. Report an issue: GitHub.