{"record":{"id":"8c21cdc1c64e20f2","repo":"apache/cassandra","slug":"bloom-filter-size-is-16gb-reduce-the-bloom-filt","errorCode":null,"errorMessage":"Bloom filter size is > 16GB, reduce the bloom_filter_fp_chance","messagePattern":"Bloom filter size is > 16GB, reduce the bloom_filter_fp_chance","errorType":"exception","errorClass":"UnsupportedOperationException","httpStatus":null,"severity":"error","filePath":"src/java/org/apache/cassandra/utils/obs/OffHeapBitSet.java","lineNumber":46,"sourceCode":"import org.apache.cassandra.io.util.Memory;\nimport org.apache.cassandra.io.util.MemoryOutputStream;\nimport org.apache.cassandra.utils.FBUtilities;\nimport org.apache.cassandra.utils.concurrent.Ref;\n\n/**\n * Off-heap bitset,\n * file compatible with OpeBitSet\n */\npublic class OffHeapBitSet implements IBitSet\n{\n    private final Memory bytes;\n\n    public OffHeapBitSet(long numBits)\n    {\n        /** returns the number of 64 bit words it would take to hold numBits */\n        long wordCount = (((numBits - 1) >>> 6) + 1);\n        if (wordCount > Integer.MAX_VALUE)\n            throw new UnsupportedOperationException(\"Bloom filter size is > 16GB, reduce the bloom_filter_fp_chance\");\n        try\n        {\n            long byteCount = wordCount * 8L;\n            bytes = Memory.allocate(byteCount);\n        }\n        catch (OutOfMemoryError e)\n        {\n            throw new RuntimeException(\"Out of native memory occured, You can avoid it by increasing the system ram space or by increasing bloom_filter_fp_chance.\");\n        }\n        // flush/clear the existing memory.\n        clear();\n    }\n\n    private OffHeapBitSet(Memory bytes)\n    {\n        this.bytes = bytes;\n    }\n","sourceCodeStart":28,"sourceCodeEnd":64,"githubUrl":"https://github.com/apache/cassandra/blob/88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1/src/java/org/apache/cassandra/utils/obs/OffHeapBitSet.java#L28-L64","documentation":"OffHeapBitSet sizes its off-heap bit array from the requested bit count; if that requires more than Integer.MAX_VALUE 64-bit words (i.e. > 16GB) it refuses with UnsupportedOperationException because the word count cannot be stored in an int. The bloom filter derived from bloom_filter_fp_chance is too large.","triggerScenarios":"Creating a bloom filter whose requested numBits implies wordCount > Integer.MAX_VALUE, e.g. building an SSTable summary/bloom filter with a very low bloom_filter_fp_chance and a huge number of keys.","commonSituations":"bloom_filter_fp_chance set extremely low (near 0.001 or below) on tables with billions of rows; default fp chance lowered to reduce false positives on huge compacted SSTables.","solutions":["Raise the table's bloom_filter_fp_chance (e.g. ALTER TABLE ... WITH bloom_filter_fp_chance = 0.01 or higher) to shrink the filter.","Reduce per-SSTable key counts by compacting more or partitioning data across tables/keyspaces.","If the fp chance is required, use a non-off-heap or large-capacity filter implementation available in your version."],"exampleFix":"// before\nALTER TABLE big_table WITH bloom_filter_fp_chance = 0.0005;\n// after\nALTER TABLE big_table WITH bloom_filter_fp_chance = 0.01;","handlingStrategy":"validation","validationCode":"double fpChance = tableParams.getBloomFilterFpChance();\nif (fpChance < 0.01) logger.warn(\"bloom_filter_fp_chance {} very low; filter may exceed 16GB on large tables\", fpChance);","typeGuard":null,"tryCatchPattern":"try { new OffHeapBitSet(numBits); } catch (UnsupportedOperationException e) { /* lower filter size or raise fp chance */ }","preventionTips":["Keep bloom_filter_fp_chance within the supported range (default 0.01, floor 0.00055...).","Estimate filter size: bits ~= -keys * ln(fp) / (ln 2)^2 before creating huge filters."],"tags":["bloom-filter","memory","configuration","cassandra"],"backgroundTag":"value-out-of-range","analyzedSha":"88fd0f6a0eaed8943f05ac9e8f947882b8ddc8f1","analyzedAt":"2026-09-10T07:29:22.284Z","contentChangedAt":"2026-09-10T07:29:22.284Z","schemaVersion":2},"datasetVersion":"2026-09-14T16:17:12.679Z"}