{"record":{"id":"811fa83b6bd9c21a","repo":"apache/cassandra","slug":"deserialized-partition-size-histogram-with-valu","errorCode":null,"errorMessage":"Deserialized partition size histogram with {} values greater than the maximum of {}. Clearing the overflow bucket to allow for degraded mean and percentile calculations...","messagePattern":"Deserialized partition size histogram with (.+?) values greater than the maximum of (.+?)\\. Clearing the overflow bucket to allow for degraded mean and percentile calculations\\.\\.\\.","errorType":"console","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"src/java/org/apache/cassandra/io/sstable/metadata/StatsMetadata.java","lineNumber":530,"sourceCode":"        }\n\n        private void serializeImprovedMinMax(Version version, StatsMetadata component, DataOutputPlus out) throws IOException\n        {\n            assert component.clusteringTypes != null;\n            typeSerializer.serializeList(component.clusteringTypes, out);\n            Slice.serializer.serialize(component.coveredClustering,\n                                       out,\n                                       version.correspondingMessagingVersion(),\n                                       component.clusteringTypes);\n        }\n\n        public StatsMetadata deserialize(Version version, DataInputPlus in) throws IOException\n        {\n            EstimatedHistogram partitionSizes = EstimatedHistogram.serializer.deserialize(in);\n\n            if (partitionSizes.isOverflowed())\n            {\n                logger.warn(\"Deserialized partition size histogram with {} values greater than the maximum of {}. \" +\n                            \"Clearing the overflow bucket to allow for degraded mean and percentile calculations...\",\n                            partitionSizes.overflowCount(), partitionSizes.getLargestBucketOffset());\n\n                partitionSizes.clearOverflow();\n            }\n\n            EstimatedHistogram columnCounts = EstimatedHistogram.serializer.deserialize(in);\n\n            if (columnCounts.isOverflowed())\n            {\n                logger.warn(\"Deserialized partition cell count histogram with {} values greater than the maximum of {}. \" +\n                            \"Clearing the overflow bucket to allow for degraded mean and percentile calculations...\",\n                            columnCounts.overflowCount(), columnCounts.getLargestBucketOffset());\n\n                columnCounts.clearOverflow();\n            }\n\n            CommitLogPosition commitLogLowerBound = CommitLogPosition.NONE, commitLogUpperBound;","sourceCodeStart":512,"sourceCodeEnd":548,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/io/sstable/metadata/StatsMetadata.java#L512-L548","documentation":"When deserializing an sstable's StatsMetadata, the partition-size EstimatedHistogram reports that its overflow bucket holds values beyond the largest representable bucket offset. Cassandra logs a warning and clears the overflow bucket so mean/percentile math does not blow up, at the cost of accuracy for the largest partitions.","triggerScenarios":"Reading the StatsMetadata component of an sstable whose partition sizes exceeded the histogram's maximum tracked value when it was written (e.g. very large partitions, or histograms written by older/other versions).","commonSituations":"Upgrading clusters with very large partitions; restoring sstables from clusters with different partition-size distributions; corrupted or partially-written -Statistics.db components.","solutions":["Verify partition sizes with `nodetool tablehistograms` and consider `nodetool compact` to rewrite the sstable","Check for extremely large partitions (`nodetool toppartitions` or scanning for big rows) and split them via a better partition key","If only one sstable is affected and data is intact, treat as informational; metrics are degraded but correctness is unaffected","If corruption is suspected, run a full repair/rebuild of the affected table from other replicas"],"exampleFix":"null","handlingStrategy":"validation","validationCode":"EstimatedHistogram sizes = StatsMetadata.serializer.deserialize(version, in);\nif (sizes.isOverflowed()) {\n    logger.warn(\"Partition size histogram overflowed: {} values beyond max {}\", sizes.overflowCount(), sizes.getLargestBucketOffset());\n    sizes.clearOverflow(); // degraded stats, data still safe\n}","typeGuard":"boolean isHistogramUsable(EstimatedHistogram h) { return h != null && !h.isOverflowed(); }","tryCatchPattern":"try { StatsMetadata sm = StatsMetadata.serializer.deserialize(version, in); } catch (IOException e) { /* corrupted -Statistics.db: quarantine sstable, rebuild from replicas */ }","preventionTips":["Avoid partitions large enough to overflow histogram buckets","Run repairs so corrupt sstables are replaced by healthy replicas","Monitor nodetool tablehistograms for overflow indicators"],"tags":["sstable","histogram","io","deserialization"],"backgroundTag":"value-out-of-range","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}