prestodb/presto · error · IllegalArgumentException

BloomFilters are not compatible for merging. this -

Error message

BloomFilters are not compatible for merging. this - 

What it means

BloomFilter.merge() ORs another BloomFilter's bitset into this one, but only when both filters share identical numBits and numHashFunctions (and are not the same object); otherwise it throws IllegalArgumentException. Merging incompatible bloom filters would produce statistically meaningless results, so the library rejects it.

Source

Thrown at presto-orc/src/main/java/com/facebook/presto/orc/metadata/statistics/BloomFilter.java:201

    }

    public long[] getBitSet()
    {
        return this.bitSet.getData();
    }

    public String toString()
    {
        return "m: " + this.numBits + " k: " + this.numHashFunctions;
    }

    public void merge(BloomFilter that)
    {
        if (this != that && this.numBits == that.numBits && this.numHashFunctions == that.numHashFunctions) {
            this.bitSet.putAll(that.bitSet);
        }
        else {
            throw new IllegalArgumentException("BloomFilters are not compatible for merging. this - " + this.toString() + " that - " + that.toString());
        }
    }

    public void reset()
    {
        this.bitSet.clear();
    }

    public static class BitSet
    {
        private final long[] data;

        public BitSet(long bits)
        {
            this(new long[(int) Math.ceil((double) bits / 64.0D)]);
        }

        public BitSet(long[] data)

View on GitHub (pinned to 55bb57d202)

Solutions

1. Align bloom filter config (bloomFilterFpp / bloomFilterExpectedSize) across all writers producing the files
2. Check numBits/numHashFunctions of both filters before merging and skip or rebuild incompatible ones
3. Recompute the merged bloom filter from raw data instead of merging mismatched filters

Example fix

// before
first.merge(second); // throws if params differ
// after
if (first.getNumBits() == second.getNumBits() && first.getNumHashFunctions() == second.getNumHashFunctions()) {
    first.merge(second);
} else {
    BloomFilter rebuilt = new BloomFilter(targetExpectedEntries, targetFpp);
    rebuilt.merge(first); rebuilt.merge(second);
}

// Check compatibility before merging
public void safeMerge(BloomFilter target, BloomFilter other) {
    checkArgument(target.getNumBits() == other.getNumBits(), "numBits mismatch: %s vs %s", target.getNumBits(), other.getNumBits());
    checkArgument(target.getNumHashFunctions() == other.getNumHashFunctions(), "numHashFunctions mismatch");
    target.merge(other);
}

try {
    merged.merge(that);
} catch (IllegalArgumentException e) {
    if (e.getMessage() != null && e.getMessage().startsWith("BloomFilters are not compatible")) {
        // rebuild from source data or skip merge
        merged = new BloomFilter(expectedEntries, fpp);
    } else { throw e; }
}

When it happens

Trigger: Merging bloom filter statistics from ORC files/stripe metadata whose parameters (numBits, numHashFunctions) differ — e.g. files written with different bloom filter fpp or size settings.

Common situations: Compacting/merging ORC files written with different writer configs; schema evolution changing bloom filter settings; aggregating stats across mixed-version writers.

Related errors

• NOT_SUPPORTED
• HIVE_WRITER_CLOSE_ERROR
• HIVE_WRITER_OPEN_ERROR
• HIVE_UNSUPPORTED_FORMAT
• HIVE_INVALID_METADATA

AI-assisted analysis of prestodb/presto@55bb57d202 (2026-09-04). Data as JSON: /api/errors/d01b13b4efae909f. Report an issue: GitHub.