{"record":{"id":"d01b13b4efae909f","repo":"prestodb/presto","slug":"bloomfilters-are-not-compatible-for-merging-this","errorCode":null,"errorMessage":"BloomFilters are not compatible for merging. this - ","messagePattern":"BloomFilters are not compatible for merging\\. this - ","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"presto-orc/src/main/java/com/facebook/presto/orc/metadata/statistics/BloomFilter.java","lineNumber":201,"sourceCode":"    }\n\n    public long[] getBitSet()\n    {\n        return this.bitSet.getData();\n    }\n\n    public String toString()\n    {\n        return \"m: \" + this.numBits + \" k: \" + this.numHashFunctions;\n    }\n\n    public void merge(BloomFilter that)\n    {\n        if (this != that && this.numBits == that.numBits && this.numHashFunctions == that.numHashFunctions) {\n            this.bitSet.putAll(that.bitSet);\n        }\n        else {\n            throw new IllegalArgumentException(\"BloomFilters are not compatible for merging. this - \" + this.toString() + \" that - \" + that.toString());\n        }\n    }\n\n    public void reset()\n    {\n        this.bitSet.clear();\n    }\n\n    public static class BitSet\n    {\n        private final long[] data;\n\n        public BitSet(long bits)\n        {\n            this(new long[(int) Math.ceil((double) bits / 64.0D)]);\n        }\n\n        public BitSet(long[] data)","sourceCodeStart":183,"sourceCodeEnd":219,"githubUrl":"https://github.com/prestodb/presto/blob/55bb57d202de3b926896fa966c2c4a44c779634e/presto-orc/src/main/java/com/facebook/presto/orc/metadata/statistics/BloomFilter.java#L183-L219","documentation":"BloomFilter.merge() ORs another BloomFilter's bitset into this one, but only when both filters share identical numBits and numHashFunctions (and are not the same object); otherwise it throws IllegalArgumentException. Merging incompatible bloom filters would produce statistically meaningless results, so the library rejects it.","triggerScenarios":"Merging bloom filter statistics from ORC files/stripe metadata whose parameters (numBits, numHashFunctions) differ — e.g. files written with different bloom filter fpp or size settings.","commonSituations":"Compacting/merging ORC files written with different writer configs; schema evolution changing bloom filter settings; aggregating stats across mixed-version writers.","solutions":["Align bloom filter config (bloomFilterFpp / bloomFilterExpectedSize) across all writers producing the files","Check numBits/numHashFunctions of both filters before merging and skip or rebuild incompatible ones","Recompute the merged bloom filter from raw data instead of merging mismatched filters"],"exampleFix":"// before\nfirst.merge(second); // throws if params differ\n// after\nif (first.getNumBits() == second.getNumBits() && first.getNumHashFunctions() == second.getNumHashFunctions()) {\n    first.merge(second);\n} else {\n    BloomFilter rebuilt = new BloomFilter(targetExpectedEntries, targetFpp);\n    rebuilt.merge(first); rebuilt.merge(second);\n}","handlingStrategy":"validation","validationCode":"// Check compatibility before merging\npublic void safeMerge(BloomFilter target, BloomFilter other) {\n    checkArgument(target.getNumBits() == other.getNumBits(), \"numBits mismatch: %s vs %s\", target.getNumBits(), other.getNumBits());\n    checkArgument(target.getNumHashFunctions() == other.getNumHashFunctions(), \"numHashFunctions mismatch\");\n    target.merge(other);\n}","typeGuard":null,"tryCatchPattern":"try {\n    merged.merge(that);\n} catch (IllegalArgumentException e) {\n    if (e.getMessage() != null && e.getMessage().startsWith(\"BloomFilters are not compatible\")) {\n        // rebuild from source data or skip merge\n        merged = new BloomFilter(expectedEntries, fpp);\n    } else { throw e; }\n}","preventionTips":["Use identical bloomFilterFpp and bloomFilterExpectedSize across all writers","Never change bloom filter settings between compaction generations","Store bloom parameters alongside stats and compare before merge"],"tags":["orc","bloom-filter","statistics-merge","incompatible-parameters"],"backgroundTag":"bloom-filter-incompatible-merge","analyzedSha":"55bb57d202de3b926896fa966c2c4a44c779634e","analyzedAt":"2026-09-04T12:50:26.162Z","contentChangedAt":"2026-09-04T12:50:26.162Z","schemaVersion":2},"datasetVersion":"2026-09-11T21:17:09.523Z"}