apache/cassandra · warning

Out of order partition detected

Error message

Out of order partition detected (%s found after %s)

What it means

saveOutOfOrderPartition is called by tryAppend when scrubbing encounters a partition whose decorated key sorts before the previously appended key — a violation of the sstable's key-order invariant. The partition is warned about and buffered into outOfOrder to be written to a separate sstable rather than merged into the main output.

Solutions

1. Accept the automatic mitigation: the partition is preserved in the out-of-order sstable created by the scrubber
2. Run nodetool repair to re-sync with healthy replicas
3. Audit how the sstable was produced (restore tooling, version) to find the source of the ordering violation
4. Check storage integrity if multiple sstables show the same symptom

When it happens

Trigger: Scrub iterates partitions in file order; a key compares less than or equal to the previous key (prevKey), violating strictly increasing decorated-key order, so saveOutOfOrderPartition fires.

Common situations: Corrupted index/data pointing mid-file; sstables written by buggy older versions; botched manual copying/restoring of sstable components; disk corruption altering a key.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• out of order partition (or partitions without of order…
• Duplicate row detected in
• Out of order rows found in partition
• Data file partition position
• Data file partition position

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/da6d5097d1d91eaf. Report an issue: GitHub.

Appendix: source

Thrown at src/java/org/apache/cassandra/io/sstable/format/SortedTableScrubber.java:338

     */
    private UnfilteredRowIterator getIterator(DecoratedKey key)
    {
        RowMergingSSTableIterator rowMergingIterator = new RowMergingSSTableIterator(SSTableIdentityIterator.create(sstable,
                                                                                                                    dataFile,
                                                                                                                    key),
                                                                                     outputHandler,
                                                                                     sstable.descriptor.version,
                                                                                     options.reinsertOverflowedTTLRows);
        if (options.reinsertOverflowedTTLRows)
            return new FixNegativeLocalDeletionTimeIterator(rowMergingIterator, outputHandler, negativeLocalDeletionInfoMetrics);
        else
            return rowMergingIterator;
    }

    private void saveOutOfOrderPartition(DecoratedKey prevKey, DecoratedKey key, UnfilteredRowIterator iterator)
    {
        // TODO bitch if the row is too large?  if it is there's not much we can do ...
        outputHandler.warn("Out of order partition detected (%s found after %s)", keyString(key), keyString(prevKey));
        outOfOrder.add(ImmutableBTreePartition.create(iterator));
    }

    protected static void throwIfFatal(Throwable th)
    {
        if (th instanceof Error && !(th instanceof AssertionError || th instanceof IOError))
            throw (Error) th;
    }

    protected void throwIfCannotContinue(DecoratedKey key, Throwable th)
    {
        if (isCommutative && !options.skipCorrupted)
        {
            outputHandler.warn("An error occurred while scrubbing the partition with key '%s'.  Skipping corrupt " +
                               "data in counter tables will result in undercounts for the affected " +
                               "counters (see CASSANDRA-2759 for more details), so by default the scrub will " +
                               "stop at this point.  If you would like to skip the row anyway and continue " +
                               "scrubbing, re-run the scrub with the --skip-corrupted option.",

View on GitHub (pinned to 88fd0f6a0e)