apache/hadoop · error · IllegalArgumentException

filters cannot be or-ed

Error message

filters cannot be or-ed

What it means

BloomFilter.or(Filter) unions another filter's bits into this one. Like and() and xor(), it requires the operand to be a non-null BloomFilter with exactly the same vectorSize and nbHash; a bit-union only makes sense between filters over the same bit space and hash functions, so anything else throws IllegalArgumentException('filters cannot be or-ed').

Source

Thrown at hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/bloom/BloomFilter.java:169

      if(!bits.get(h[i])) {
        return false;
      }
    }
    return true;
  }

  @Override
  public void not() {
    bits.flip(0, vectorSize);
  }

  @Override
  public void or(Filter filter) {
    if(filter == null
        || !(filter instanceof BloomFilter)
        || filter.vectorSize != this.vectorSize
        || filter.nbHash != this.nbHash) {
      throw new IllegalArgumentException("filters cannot be or-ed");
    }
    bits.or(((BloomFilter) filter).bits);
  }

  @Override
  public void xor(Filter filter) {
    if(filter == null
        || !(filter instanceof BloomFilter)
        || filter.vectorSize != this.vectorSize
        || filter.nbHash != this.nbHash) {
      throw new IllegalArgumentException("filters cannot be xor-ed");
    }
    bits.xor(((BloomFilter) filter).bits);
  }

  @Override
  public String toString() {
    return bits.toString();

View on GitHub (pinned to 2add963021)

Solutions

1. Create all union participants with the same constructor parameters — centralize vectorSize/nbHash/hashType in shared constants.
2. Validate shape equality before calling or() and drop/rebuild mismatched filters.
3. If union participants legitimately differ, add all underlying elements into one uniformly shaped filter instead.
4. Catch IllegalArgumentException during aggregation to isolate the misconfigured producer.

Example fix

// before
BloomFilter merged = new BloomFilter(VECTOR_SIZE, HASH_COUNT, Hash.MURMUR_HASH);
for (BloomFilter f : shardFilters) { // one shard used VECTOR_SIZE 512
  merged.or(f); // throws: filters cannot be or-ed
}

// after
for (BloomFilter f : shardFilters) {
  if (f.vectorSize == merged.vectorSize && f.nbHash == merged.nbHash) {
    merged.or(f);
  }
}

for (BloomFilter f : shardFilters) {
  if (f == null || f.vectorSize != merged.vectorSize
      || f.nbHash != merged.nbHash) {
    LOG.warn("Skipping shape-mismatched filter");
    continue;
  }
  merged.or(f);
}

static boolean sameShape(BloomFilter a, BloomFilter b) {
  return b != null && b.vectorSize == a.vectorSize && b.nbHash == a.nbHash;
}

try {
  merged.or(f);
} catch (IllegalArgumentException e) {
  throw new IllegalStateException("Union aborted: producer filter shape="
      + f.vectorSize + "/" + f.nbHash, e);
}

When it happens

Trigger: filterA.or(filterB) where the two filters were constructed with different vectorSize or nbHash; filterA.or(null); filterA.or(new Filter subclass) that is not a BloomFilter.

Common situations: Aggregating per-block or per-node bloom filters (a classic union use) where producers were configured with different sizes; rolling upgrades changing the default vector size between filter producers; hard-coded constructor literals drifting apart between classes.

Related errors

• filters cannot be and-ed
• filters cannot be xor-ed
• filters cannot be and-ed
• Input files cannot be merged as they have different Key and
• Input files cannot be merged as they have different Key clas

AI-assisted analysis of apache/hadoop@2add963021 (2026-08-22). Data as JSON: /api/errors/a52b3a6d7750f71d. Report an issue: GitHub.