{"record":{"id":"b26ad5febb3ee2b5","repo":"apache/hadoop","slug":"filters-cannot-be-and-ed","errorCode":null,"errorMessage":"filters cannot be and-ed","messagePattern":"filters cannot be and-ed","errorType":"exception","errorClass":"IllegalArgumentException","httpStatus":null,"severity":"error","filePath":"hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/bloom/BloomFilter.java","lineNumber":136,"sourceCode":"    if(key == null) {\n      throw new NullPointerException(\"key cannot be null\");\n    }\n\n    int[] h = hash.hash(key);\n    hash.clear();\n\n    for(int i = 0; i < nbHash; i++) {\n      bits.set(h[i]);\n    }\n  }\n\n  @Override\n  public void and(Filter filter) {\n    if(filter == null\n        || !(filter instanceof BloomFilter)\n        || filter.vectorSize != this.vectorSize\n        || filter.nbHash != this.nbHash) {\n      throw new IllegalArgumentException(\"filters cannot be and-ed\");\n    }\n\n    this.bits.and(((BloomFilter) filter).bits);\n  }\n\n  @Override\n  public boolean membershipTest(Key key) {\n    if(key == null) {\n      throw new NullPointerException(\"key cannot be null\");\n    }\n\n    int[] h = hash.hash(key);\n    hash.clear();\n    for(int i = 0; i < nbHash; i++) {\n      if(!bits.get(h[i])) {\n        return false;\n      }\n    }","sourceCodeStart":118,"sourceCodeEnd":154,"githubUrl":"https://github.com/apache/hadoop/blob/2add9630210752f88ceb1bb74eb65e37bf41da8e/hadoop-common-project/hadoop-common/src/main/java/org/apache/hadoop/util/bloom/BloomFilter.java#L118-L154","documentation":"BloomFilter.and(Filter) merges another filter's bits into this one and requires that the operand is a BloomFilter (not null, not a subclass of another Filter type) and was constructed with the identical vectorSize and nbHash. Merging differently shaped filters would be meaningless (different bit universes, different hash count), so mismatches throw IllegalArgumentException with this message.","triggerScenarios":"filterA.and(null); filterA.and(someRemoveFilter); filterA.and(filterB) where filterB was built with new BloomFilter(640, 8, Hash.JENKINS_HASH) but filterA used vectorSize 1024 — any vectorSize or nbHash difference between the two constructors triggers it.","commonSituations":"Merging per-shard or per-worker bloom filters that were configured from different settings; one component defaulting to a different vector size after a config change; aggregating filters built at different code versions with different defaults.","solutions":["Construct all filters you intend to combine with identical vectorSize and nbHash, ideally from one shared configuration constant.","Check compatibility before combining: filter != null && filter instanceof BloomFilter && filter.vectorSize == this.vectorSize && filter.nbHash == this.nbHash.","If shapes differ, rebuild one side or re-add elements from the raw data instead of merging bitsets.","Catch IllegalArgumentException around merge loops to identify the mismatched producer."],"exampleFix":"// before\nBloomFilter a = new BloomFilter(1024, 4, Hash.MURMUR_HASH);\nBloomFilter b = new BloomFilter(640, 4, Hash.MURMUR_HASH);\na.and(b); // throws: filters cannot be and-ed\n\n// after\nBloomFilter b = new BloomFilter(1024, 4, Hash.MURMUR_HASH); // same shape\na.and(b);","handlingStrategy":"validation","validationCode":"static boolean isMergeable(BloomFilter target, Filter other) {\n  return other instanceof BloomFilter\n      && other.vectorSize == target.vectorSize\n      && other.nbHash == target.nbHash;\n}\n\nif (!isMergeable(target, other)) {\n  throw new IllegalStateException(\n      \"Cannot merge filter with shape \" + shapeOf(other)\n      + \" into \" + shapeOf(target));\n}\ntarget.and(other);","typeGuard":"static boolean isMergeableBloomFilter(Filter f, int vectorSize, int nbHash) {\n  return f != null\n      && f instanceof BloomFilter\n      && f.vectorSize == vectorSize\n      && f.nbHash == nbHash;\n}","tryCatchPattern":"try {\n  target.and(other);\n} catch (IllegalArgumentException e) {\n  LOG.error(\"Filter shape mismatch during AND; rebuild from raw data\", e);\n  rebuildFromSource(target);\n}","preventionTips":["Construct all merge-participant filters from one shared shape constant.","Log vectorSize/nbHash in filter-producing components for diagnosability.","Check shape on deserialized filters before combining."],"tags":["bloom-filter","incompatible-parameters","merge","hadoop-common"],"backgroundTag":"bloom-filter-incompatible","analyzedSha":"2add9630210752f88ceb1bb74eb65e37bf41da8e","analyzedAt":"2026-08-22T19:55:07.957Z","schemaVersion":2},"datasetVersion":"2026-08-23T01:17:44.959Z"}