apache/druid · error · ISE

Unexpected null value in BloomFilterMergeAggregator

Error message

Unexpected null value in BloomFilterMergeAggregator

What it means

BloomFilterMergeAggregator.bufferAdd() reads a ByteBuffer (serialized bloom filter) from the underlying selector and merges it into the buffer. By contract, null columns should already be represented as empty bloom filters at this stage, so a null from the selector indicates a broken invariant and the aggregator throws IllegalStateException.

Source

Thrown at extensions-core/druid-bloom-filter/src/main/java/org/apache/druid/query/aggregation/bloom/BloomFilterMergeAggregator.java:42

import org.apache.druid.segment.BaseObjectColumnValueSelector;

import java.nio.ByteBuffer;

public final class BloomFilterMergeAggregator
    extends BaseBloomFilterAggregator<BaseObjectColumnValueSelector<ByteBuffer>>
{
  BloomFilterMergeAggregator(BaseObjectColumnValueSelector<ByteBuffer> selector, int maxNumEntries, boolean onHeap)
  {
    super(selector, maxNumEntries, onHeap);
  }

  @Override
  public void bufferAdd(ByteBuffer buf)
  {
    ByteBuffer other = selector.getObject();
    if (other == null) {
      // nulls should be empty bloom filters by this point, so encountering a nil column in merge agg is unexpected
      throw new ISE("Unexpected null value in BloomFilterMergeAggregator");
    }
    BloomKFilter.mergeBloomFilterByteBuffers(buf, buf.position(), other, other.position());
  }
}

View on GitHub (pinned to 9b90983fd2)

Solutions

Ensure the aggregation path produces empty BloomKFilter buffers instead of nulls (serialize an empty BloomKFilter with matching numEntries) before merging
Filter out rows with null values or use a fallback selector that substitutes an empty serialized bloom filter
Verify the field dimension exists in all segments/results being merged
Use matching maxNumEntries across producing and merging aggregators so buffers are compatible

Example fix

// before
ByteBuffer other = selector.getObject(); // null -> ISE
// after
ByteBuffer other = selector.getObject();
if (other == null) {
  other = ByteBuffer.wrap(new BloomKFilter(maxNumEntries).toByteBuffer().array()); // empty filter
}
BloomKFilter.mergeBloomFilterByteBuffers(buf, buf.position(), other, other.position());

Defensive patterns

Strategy: type-guard

Validate before calling

// Before merge: ensure every intermediate value is a non-null serialized bloom filter
boolean mergeable(Object v) { return v instanceof ByteBuffer && ((ByteBuffer) v).remaining() > 0; }

Type guard

if (selector instanceof NilColumnValueSelector) throw ...; // checked upstream; also guard getObject() == null

Try / catch

try {
  agg.bufferAdd(buf);
} catch (IllegalStateException e) {
  if (e.getMessage().contains("Unexpected null value")) {
    // substitute an empty serialized BloomKFilter and skip this row
  } else throw e;
}

Prevention

Ensure upstream aggregation always emits empty filters instead of nulls
Confirm the field exists in every segment/subquery result being merged
Match maxNumEntries across producers and mergers

When it happens

Trigger: Running a query with the merge aggregator (bloom filter merge, e.g. subquery/group-by merge path) where the selector is a nil selector returning null for every row — e.g. the column is entirely absent, or the merge path was fed raw nulls without prior conversion to empty filters.

Common situations: Distributed query merge where one broker/segment produced null intermediate values; combining bloom results in subtotals/subqueries where the column is missing in a partial result; extension/custom code invoking bufferAdd on a selector that returns null objects.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/c8b42549d356b7e6. Report an issue: GitHub.