What causes "Failed to deserialize bloom filter" in apache/druid?

Regenerate the bloom filter bytes using Druid's BloomKFilter together with BloomFilterSerializersModule/ObjectMapper serialization and re-encode as base64.

apache/druid · error · RuntimeException

Failed to deserialize bloom filter

Error message

Failed to deserialize bloom filter

What it means

During SQL-to-native translation of BLOOM_FILTER(x, filter), the base64-decoded filter bytes are converted back into a BloomKFilter. If that byte payload cannot be deserialized as a bloom filter (IOException), the operator wraps it in a RuntimeException with this message. It indicates the filter literal in the SQL statement is corrupt or not a valid bloom filter serialization.

Solutions

Regenerate the bloom filter bytes using Druid's BloomKFilter together with BloomFilterSerializersModule/ObjectMapper serialization and re-encode as base64.
Validate the base64 string is complete and unchanged (no whitespace/newline truncation, correct URL-safety) before embedding it in the SQL.
Check client and Druid versions for serialization compatibility; upgrade the client or re-serialize with the server's format if versions differ.

Example fix

// before
String filter = base64FromLegacyGuavaBloomFilter; // wrong format

// after
ObjectMapper mapper = new ObjectMapper();
mapper.registerModule(new BloomFilterSerializersModule());
BloomKFilter f = BloomFilterSerializersModule.bloomKFilterFromBytes(
    Base64.getDecoder().decode(generatedByDruid));
String filter = Base64.getEncoder().encodeToString(
    BloomFilterSerializersModule.bloomKFilterToBytes(f));

Defensive patterns

Strategy: try-catch

Validate before calling

// Validate round-trip before sending bytes into SQL
BloomKFilter check = BloomFilterSerializersModule.bloomKFilterFromBytes(
    Base64.getDecoder().decode(base64Filter)); // throws early if corrupt

Try / catch

try {
  BloomKFilter f = BloomFilterSerializersModule.bloomKFilterFromBytes(decoded);
} catch (IOException | IllegalArgumentException e) {
  throw new ISE("Bloom filter literal is not valid BloomKFilter bytes: %s", e.getMessage());
}

Prevention

Generate filter strings only via Druid's ObjectMapper with BloomFilterSerializersModule registered.
Base64-encode/decode round-trip test the filter string before embedding it in SQL.
Avoid passing filter strings through tools that reflow or truncate whitespace.

When it happens

Trigger: Passing a malformed or truncated base64 string as the second argument of BLOOM_FILTER(); decoding succeeds but the bytes are not a BloomKFilter stream; a producer serialized the filter with an incompatible format/version.

Common situations: Hand-copying a filter string and accidentally truncating it; using a bloom filter generated by a different library (e.g. Hadoop/Guava bloom filter) instead of Druid's BloomKFilter serialization; version skew between client-side serializer and server deserializer after a Druid upgrade.

Understand the failure class

Background: "failed to unmarshal" / json.Unmarshal errors: why parsing a response into a Go struct fails and how to fix it — this error's family across 23 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/cf7554a267361a24. Report an issue: GitHub.

Appendix: source

Thrown at extensions-core/druid-bloom-filter/src/main/java/org/apache/druid/query/filter/sql/BloomFilterOperatorConversion.java:88

    final DruidExpression druidExpression = Expressions.toDruidExpression(
        plannerContext,
        rowSignature,
        operands.get(0)
    );
    if (druidExpression == null) {
      return null;
    }

    String base64EncodedBloomKFilter = RexLiteral.stringValue(operands.get(1));
    final byte[] decoded = StringUtils.decodeBase64String(base64EncodedBloomKFilter);
    BloomKFilter filter;
    BloomKFilterHolder holder;
    try {
      filter = BloomFilterSerializersModule.bloomKFilterFromBytes(decoded);
      holder = BloomKFilterHolder.fromBloomKFilter(filter);
    }
    catch (IOException ioe) {
      throw new RuntimeException("Failed to deserialize bloom filter", ioe);
    }

    if (druidExpression.isSimpleExtraction()) {
      return new BloomDimFilter(
          druidExpression.getSimpleExtraction().getColumn(),
          holder,
          druidExpression.getSimpleExtraction().getExtractionFn(),
          null
      );
    } else if (virtualColumnRegistry != null) {
      String virtualColumnName = virtualColumnRegistry.getOrCreateVirtualColumnForExpression(
          druidExpression,
          operands.get(0).getType()
      );
      if (virtualColumnName == null) {
        return null;
      }
      return new BloomDimFilter(

View on GitHub (pinned to 9b90983fd2)