{"record":{"id":"40d8b12044c3ab02","repo":"apache/cassandra","slug":"detected-duplicate-rows-for-during","errorCode":null,"errorMessage":"Detected {} duplicate rows for {} during {}.{}","messagePattern":"Detected (.+?) duplicate rows for (.+?) during (.+?)\\.(.+?)","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"src/java/org/apache/cassandra/db/transform/DuplicateRowChecker.java","lineNumber":102,"sourceCode":"    }\n\n    protected Row applyToRow(Row row)\n    {\n        if (null != previous && metadata.comparator.compare(row.clustering(), previous) == 0)\n        {\n            duplicatesDetected++;\n            hadNonEqualDuplicates |= !row.clustering().equals(previous);\n        }\n\n        previous = row.clustering();\n        return row;\n    }\n\n    protected void onPartitionClose()\n    {\n        if (duplicatesDetected > 0)\n        {\n            logger.warn(\"Detected {} duplicate rows for {} during {}.{}\",\n                        duplicatesDetected,\n                        metadata.partitionKeyType.getString(key.getKey()),\n                        stage,\n                        hadNonEqualDuplicates ? \" Some duplicates had different byte representation.\" : \"\");\n            if (snapshotOnDuplicate)\n                DiagnosticSnapshotService.duplicateRows(metadata, replicas);\n        }\n        duplicatesDetected = 0;\n        previous = null;\n        super.onPartitionClose();\n    }\n\n    public static UnfilteredPartitionIterator duringCompaction(final UnfilteredPartitionIterator iterator, OperationType type)\n    {\n        if (!DatabaseDescriptor.checkForDuplicateRowsDuringCompaction())\n            return iterator;\n        final List<InetAddressAndPort> address = Collections.singletonList(FBUtilities.getBroadcastAddressAndPort());\n        final boolean snapshot = DatabaseDescriptor.snapshotOnDuplicateRowDetection();","sourceCodeStart":84,"sourceCodeEnd":120,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/db/transform/DuplicateRowChecker.java#L84-L120","documentation":"DuplicateRowChecker is a diagnostics transform that detects duplicate row Clusters within a partition as data passes through read/compaction/verification pipelines. At partition close, if duplicates were seen, it warns with the count, partition key, and stage; if snapshot-on-duplicate diagnostics is enabled it also captures a diagnostic snapshot. It indicates possible on-disk corruption or a compaction bug, not a user error.","triggerScenarios":"onPartitionClose() runs after iterating a partition and duplicatesDetected > 0; rows with identical (or differing-byte but same-clustering) keys were observed within one partition during the given stage.","commonSituations":"Suspected disk corruption on sstables; hardware issues; rare compaction or write-path bugs; running with diagnostic snapshots enabled while investigating another issue.","solutions":["Run nodetool scrub on the affected table to remove/repair duplicate rows","Inspect the diagnostic snapshot (if taken) and the referenced sstables to identify the source","Check hardware/disk health (SMART, dmesg) on the node holding the affected sstables","Enable diagnostic snapshot on duplicates (diagnostic_snapshot_on_duplicate_rows) for future occurrences to capture evidence"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// proactively verify table integrity\n// nodetool scrub <keyspace> <table> -- to detect/repair duplicates\n// enable diagnostic_snapshot_on_duplicate_rows: true to capture evidence","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Run periodic scrub on critical tables","Monitor hardware/SMART health of storage","Enable diagnostic snapshots on duplicates in suspect environments","Investigate immediately — duplicates usually indicate corruption or a bug"],"tags":["corruption","compaction","diagnostics","data-integrity"],"backgroundTag":"duplicate-rows-detected","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}