apache/cassandra · critical · IOError

Impossible partition size (greater than file length):

Error message

Impossible partition size (greater than file length): 

What it means

Thrown by the BigTable scrubber when the partition size recorded in the SSTable index file claims the partition extends beyond the end of the data file. This means the index and data components are inconsistent — the data file is truncated or the index is corrupt — and scrubbing cannot safely continue reading that partition.

Source

Thrown at src/java/org/apache/cassandra/io/sstable/format/big/BigTableScrubber.java:164

            String keyName = key == null ? "(unreadable key)" : keyString(key);
            outputHandler.debug("partition %s is %s", keyName, FBUtilities.prettyPrintMemory(dataSizeFromIndex));
            assert currentIndexKey != null || !indexAvailable();

            try
            {
                if (key == null)
                    throw new IOError(new IOException("Unable to read partition key from data file"));

                if (currentIndexKey != null && !key.getKey().equals(currentIndexKey))
                {
                    throw new IOError(new IOException(String.format("Key from data file (%s) does not match key from index file (%s)",
                                                                    //ByteBufferUtil.bytesToHex(key.getKey()), ByteBufferUtil.bytesToHex(currentIndexKey))));
                                                                    "_too big_", ByteBufferUtil.bytesToHex(currentIndexKey))));
                }

                if (indexFile != null && dataSizeFromIndex > dataFile.length())
                    throw new IOError(new IOException("Impossible partition size (greater than file length): " + dataSizeFromIndex));

                if (indexFile != null && dataStart != dataStartFromIndex)
                    outputHandler.warn("Data file partition position %d differs from index file row position %d", dataStart, dataStartFromIndex);

                if (tryAppend(prevKey, key, writer))
                    prevKey = key;
            }
            catch (Throwable th)
            {
                throwIfFatal(th);
                outputHandler.warn(th, "Error reading partition %s (stacktrace follows):", keyName);

                if (currentIndexKey != null
                    && (key == null || !key.getKey().equals(currentIndexKey) || dataStart != dataStartFromIndex))
                {

                    outputHandler.output("Retrying from partition index; data is %s bytes starting at %s",
                                         dataSizeFromIndex, dataStartFromIndex);

View on GitHub (pinned to 88fd0f6a0e)

Solutions

1. Run `nodetool scrub` (this tool itself) or restore the SSTable from a backup/snapshot
2. Restore from backup or re-stream/repair the affected ranges (`nodetool repair` after dropping/rebuilding data, or `nodetool rebuild` from other replicas)
3. Check disk health and free space (dmesg, SMART) to rule out hardware truncation
4. Run `nodetool verify` on other SSTables to find additional corruption

// Before relying on an SSTable, verify index/data consistency
long sizeFromIndex = getIndexDataSize(descriptor); // from Index.db entry
if (sizeFromIndex > dataFile.length()) {
    // schedule scrub/repair instead of reading
    System.out.println("Truncated Data.db detected for " + descriptor + "; run nodetool scrub");
}

try { scrubber.scrub(); }
catch (IOError e) { logger.error("SSTable corrupt, scheduling repair/replacement", e); replacementSource.stream(); }

When it happens

Trigger: Running `nodetool scrub` on an SSTable whose data file is shorter than `dataSizeFromIndex` for some partition, typically after a crash mid-compaction/flush, disk-full during flush, or manual/corrupted file truncation.

Common situations: Power loss or OOM kill during flush leaving a truncated Data.db; disk full during writes; copying SSTables without all components or while being written; filesystem corruption.

Understand the failure class

Background: Checksum mismatch errors: "checksum verification failed", "digest mismatch", "expected vs actual checksum" — what they mean and how to fix them — this error's family across 41 libraries.

Related errors

• Unable to read partition key from data file
• Key from data file (%s) does not match key from index file (
• Unable to read partition key from data file
• Error reading partition %s (stacktrace follows):
• Corrupt flags value for clustering prefix (isStatic flag set

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/0d7d703f07c0b96f. Report an issue: GitHub.