{"record":{"id":"c8b42549d356b7e6","repo":"apache/druid","slug":"unexpected-null-value-in-bloomfiltermergeaggregato","errorCode":null,"errorMessage":"Unexpected null value in BloomFilterMergeAggregator","messagePattern":"Unexpected null value in BloomFilterMergeAggregator","errorType":"exception","errorClass":"ISE","httpStatus":null,"severity":"error","filePath":"extensions-core/druid-bloom-filter/src/main/java/org/apache/druid/query/aggregation/bloom/BloomFilterMergeAggregator.java","lineNumber":42,"sourceCode":"import org.apache.druid.segment.BaseObjectColumnValueSelector;\n\nimport java.nio.ByteBuffer;\n\npublic final class BloomFilterMergeAggregator\n    extends BaseBloomFilterAggregator<BaseObjectColumnValueSelector<ByteBuffer>>\n{\n  BloomFilterMergeAggregator(BaseObjectColumnValueSelector<ByteBuffer> selector, int maxNumEntries, boolean onHeap)\n  {\n    super(selector, maxNumEntries, onHeap);\n  }\n\n  @Override\n  public void bufferAdd(ByteBuffer buf)\n  {\n    ByteBuffer other = selector.getObject();\n    if (other == null) {\n      // nulls should be empty bloom filters by this point, so encountering a nil column in merge agg is unexpected\n      throw new ISE(\"Unexpected null value in BloomFilterMergeAggregator\");\n    }\n    BloomKFilter.mergeBloomFilterByteBuffers(buf, buf.position(), other, other.position());\n  }\n}\n","sourceCodeStart":24,"sourceCodeEnd":47,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/druid-bloom-filter/src/main/java/org/apache/druid/query/aggregation/bloom/BloomFilterMergeAggregator.java#L24-L47","documentation":"BloomFilterMergeAggregator.bufferAdd() reads a ByteBuffer (serialized bloom filter) from the underlying selector and merges it into the buffer. By contract, null columns should already be represented as empty bloom filters at this stage, so a null from the selector indicates a broken invariant and the aggregator throws IllegalStateException.","triggerScenarios":"Running a query with the merge aggregator (bloom filter merge, e.g. subquery/group-by merge path) where the selector is a nil selector returning null for every row — e.g. the column is entirely absent, or the merge path was fed raw nulls without prior conversion to empty filters.","commonSituations":"Distributed query merge where one broker/segment produced null intermediate values; combining bloom results in subtotals/subqueries where the column is missing in a partial result; extension/custom code invoking bufferAdd on a selector that returns null objects.","solutions":["Ensure the aggregation path produces empty BloomKFilter buffers instead of nulls (serialize an empty BloomKFilter with matching numEntries) before merging","Filter out rows with null values or use a fallback selector that substitutes an empty serialized bloom filter","Verify the field dimension exists in all segments/results being merged","Use matching maxNumEntries across producing and merging aggregators so buffers are compatible"],"exampleFix":"// before\nByteBuffer other = selector.getObject(); // null -> ISE\n// after\nByteBuffer other = selector.getObject();\nif (other == null) {\n  other = ByteBuffer.wrap(new BloomKFilter(maxNumEntries).toByteBuffer().array()); // empty filter\n}\nBloomKFilter.mergeBloomFilterByteBuffers(buf, buf.position(), other, other.position());","handlingStrategy":"type-guard","validationCode":"// Before merge: ensure every intermediate value is a non-null serialized bloom filter\nboolean mergeable(Object v) { return v instanceof ByteBuffer && ((ByteBuffer) v).remaining() > 0; }","typeGuard":"if (selector instanceof NilColumnValueSelector) throw ...; // checked upstream; also guard getObject() == null","tryCatchPattern":"try {\n  agg.bufferAdd(buf);\n} catch (IllegalStateException e) {\n  if (e.getMessage().contains(\"Unexpected null value\")) {\n    // substitute an empty serialized BloomKFilter and skip this row\n  } else throw e;\n}","preventionTips":["Ensure upstream aggregation always emits empty filters instead of nulls","Confirm the field exists in every segment/subquery result being merged","Match maxNumEntries across producers and mergers"],"tags":["druid","bloom-filter","null-value","internal-invariant"],"backgroundTag":"internal-invariant-violation","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}