apache/cassandra · warning

Detected duplicate rows for during .

Error message

Detected {} duplicate rows for {} during {}.{}

What it means

DuplicateRowChecker is a diagnostics transform that detects duplicate row Clusters within a partition as data passes through read/compaction/verification pipelines. At partition close, if duplicates were seen, it warns with the count, partition key, and stage; if snapshot-on-duplicate diagnostics is enabled it also captures a diagnostic snapshot. It indicates possible on-disk corruption or a compaction bug, not a user error.

Solutions

1. Run nodetool scrub on the affected table to remove/repair duplicate rows
2. Inspect the diagnostic snapshot (if taken) and the referenced sstables to identify the source
3. Check hardware/disk health (SMART, dmesg) on the node holding the affected sstables
4. Enable diagnostic snapshot on duplicates (diagnostic_snapshot_on_duplicate_rows) for future occurrences to capture evidence

// proactively verify table integrity
// nodetool scrub <keyspace> <table> -- to detect/repair duplicates
// enable diagnostic_snapshot_on_duplicate_rows: true to capture evidence

When it happens

Trigger: onPartitionClose() runs after iterating a partition and duplicatesDetected > 0; rows with identical (or differing-byte but same-clustering) keys were observed within one partition during the given stage.

Common situations: Suspected disk corruption on sstables; hardware issues; rare compaction or write-path bugs; running with diagnostic snapshots enabled while investigating another issue.

Related errors

• Key in . is invalid in
• Aborted garbage collection for at least one table in…
• Aborted garbage collection for at least one table, check…
• Addresses differ: !=
• At most bytes may be in a compaction level; your…

AI-assisted analysis of apache/cassandra@88fd0f6a0e (2026-09-10). Data as JSON: /api/errors/40d8b12044c3ab02. Report an issue: GitHub.

Appendix: source

Thrown at src/java/org/apache/cassandra/db/transform/DuplicateRowChecker.java:102

    }

    protected Row applyToRow(Row row)
    {
        if (null != previous && metadata.comparator.compare(row.clustering(), previous) == 0)
        {
            duplicatesDetected++;
            hadNonEqualDuplicates |= !row.clustering().equals(previous);
        }

        previous = row.clustering();
        return row;
    }

    protected void onPartitionClose()
    {
        if (duplicatesDetected > 0)
        {
            logger.warn("Detected {} duplicate rows for {} during {}.{}",
                        duplicatesDetected,
                        metadata.partitionKeyType.getString(key.getKey()),
                        stage,
                        hadNonEqualDuplicates ? " Some duplicates had different byte representation." : "");
            if (snapshotOnDuplicate)
                DiagnosticSnapshotService.duplicateRows(metadata, replicas);
        }
        duplicatesDetected = 0;
        previous = null;
        super.onPartitionClose();
    }

    public static UnfilteredPartitionIterator duringCompaction(final UnfilteredPartitionIterator iterator, OperationType type)
    {
        if (!DatabaseDescriptor.checkForDuplicateRowsDuringCompaction())
            return iterator;
        final List<InetAddressAndPort> address = Collections.singletonList(FBUtilities.getBroadcastAddressAndPort());
        final boolean snapshot = DatabaseDescriptor.snapshotOnDuplicateRowDetection();

View on GitHub (pinned to 88fd0f6a0e)