apache/druid · error · IllegalArgumentException

bf1 NumHashFunctions/NumBits does not match bf2

Error message

bf1 NumHashFunctions/NumBits does not match bf2

What it means

mergeBloomFilterBytes() can only OR together two BloomKFilter bitsets built with identical parameters. After checking equal serialized lengths, it compares the header bytes (numHashFunctions and bitset size fields before the serialized longs) and throws when they differ — the faulty inputs are two bloom filters created with different numEntries/numHashFunctions (e.g. merged from aggregators configured with different numEntries, or one side is corrupted).

Source

Thrown at extensions-core/druid-bloom-filter/src/main/java/org/apache/druid/query/filter/BloomKFilter.java:212

   * @param bf2Length
   */
  public static void mergeBloomFilterBytes(
      byte[] bf1Bytes,
      int bf1Start,
      int bf1Length,
      byte[] bf2Bytes,
      int bf2Start,
      int bf2Length
  )
  {
    if (bf1Length != bf2Length) {
      throw new IllegalArgumentException("bf1Length " + bf1Length + " does not match bf2Length " + bf2Length);
    }

    // Validation on the bitset size/3 hash functions.
    for (int idx = 0; idx < START_OF_SERIALIZED_LONGS; ++idx) {
      if (bf1Bytes[bf1Start + idx] != bf2Bytes[bf2Start + idx]) {
        throw new IllegalArgumentException("bf1 NumHashFunctions/NumBits does not match bf2");
      }
    }

    // Just bitwise-OR the bits together - size/# functions should be the same,
    // rest of the data is serialized long values for the bitset which are supposed to be bitwise-ORed.
    for (int idx = START_OF_SERIALIZED_LONGS; idx < bf1Length; ++idx) {
      bf1Bytes[bf1Start + idx] |= bf2Bytes[bf2Start + idx];
    }
  }

  public static void serialize(ByteBuffer out, BloomKFilter bloomFilter)
  {
    serialize(out, out.position(), bloomFilter);
  }

  /**
   * Serialize a bloom filter to a ByteBuffer. Does not mutate buffer position.
   *

View on GitHub (pinned to 9b90983fd2)

Solutions

1. Ensure every bloom filter aggregator in the query uses the same numEntries setting so filters share identical hash-function count and bitset size
2. Check segment metadata for bloom filters produced with mismatched or corrupt parameters and rebuild affected segments
3. If filters come from different data sources or ingestion times with different configs, merge them only after rebuilding with a common configuration

When it happens

Trigger: Thrown at extensions-core/druid-bloom-filter/src/main/java/org/apache/druid/query/filter/BloomKFilter.java:212 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/eaf8ab65cf4f1f27. Report an issue: GitHub.