{"record":{"id":"cf7554a267361a24","repo":"apache/druid","slug":"failed-to-deserialize-bloom-filter","errorCode":null,"errorMessage":"Failed to deserialize bloom filter","messagePattern":"Failed to deserialize bloom filter","errorType":"exception","errorClass":"RuntimeException","httpStatus":null,"severity":"error","filePath":"extensions-core/druid-bloom-filter/src/main/java/org/apache/druid/query/filter/sql/BloomFilterOperatorConversion.java","lineNumber":88,"sourceCode":"    final DruidExpression druidExpression = Expressions.toDruidExpression(\n        plannerContext,\n        rowSignature,\n        operands.get(0)\n    );\n    if (druidExpression == null) {\n      return null;\n    }\n\n    String base64EncodedBloomKFilter = RexLiteral.stringValue(operands.get(1));\n    final byte[] decoded = StringUtils.decodeBase64String(base64EncodedBloomKFilter);\n    BloomKFilter filter;\n    BloomKFilterHolder holder;\n    try {\n      filter = BloomFilterSerializersModule.bloomKFilterFromBytes(decoded);\n      holder = BloomKFilterHolder.fromBloomKFilter(filter);\n    }\n    catch (IOException ioe) {\n      throw new RuntimeException(\"Failed to deserialize bloom filter\", ioe);\n    }\n\n    if (druidExpression.isSimpleExtraction()) {\n      return new BloomDimFilter(\n          druidExpression.getSimpleExtraction().getColumn(),\n          holder,\n          druidExpression.getSimpleExtraction().getExtractionFn(),\n          null\n      );\n    } else if (virtualColumnRegistry != null) {\n      String virtualColumnName = virtualColumnRegistry.getOrCreateVirtualColumnForExpression(\n          druidExpression,\n          operands.get(0).getType()\n      );\n      if (virtualColumnName == null) {\n        return null;\n      }\n      return new BloomDimFilter(","sourceCodeStart":70,"sourceCodeEnd":106,"githubUrl":"https://github.com/apache/druid/blob/9b90983fd291f26935af934383ce360473179e4d/extensions-core/druid-bloom-filter/src/main/java/org/apache/druid/query/filter/sql/BloomFilterOperatorConversion.java#L70-L106","documentation":"During SQL-to-native translation of BLOOM_FILTER(x, filter), the base64-decoded filter bytes are converted back into a BloomKFilter. If that byte payload cannot be deserialized as a bloom filter (IOException), the operator wraps it in a RuntimeException with this message. It indicates the filter literal in the SQL statement is corrupt or not a valid bloom filter serialization.","triggerScenarios":"Passing a malformed or truncated base64 string as the second argument of BLOOM_FILTER(); decoding succeeds but the bytes are not a BloomKFilter stream; a producer serialized the filter with an incompatible format/version.","commonSituations":"Hand-copying a filter string and accidentally truncating it; using a bloom filter generated by a different library (e.g. Hadoop/Guava bloom filter) instead of Druid's BloomKFilter serialization; version skew between client-side serializer and server deserializer after a Druid upgrade.","solutions":["Regenerate the bloom filter bytes using Druid's BloomKFilter together with BloomFilterSerializersModule/ObjectMapper serialization and re-encode as base64.","Validate the base64 string is complete and unchanged (no whitespace/newline truncation, correct URL-safety) before embedding it in the SQL.","Check client and Druid versions for serialization compatibility; upgrade the client or re-serialize with the server's format if versions differ."],"exampleFix":"// before\nString filter = base64FromLegacyGuavaBloomFilter; // wrong format\n\n// after\nObjectMapper mapper = new ObjectMapper();\nmapper.registerModule(new BloomFilterSerializersModule());\nBloomKFilter f = BloomFilterSerializersModule.bloomKFilterFromBytes(\n    Base64.getDecoder().decode(generatedByDruid));\nString filter = Base64.getEncoder().encodeToString(\n    BloomFilterSerializersModule.bloomKFilterToBytes(f));","handlingStrategy":"try-catch","validationCode":"// Validate round-trip before sending bytes into SQL\nBloomKFilter check = BloomFilterSerializersModule.bloomKFilterFromBytes(\n    Base64.getDecoder().decode(base64Filter)); // throws early if corrupt","typeGuard":null,"tryCatchPattern":"try {\n  BloomKFilter f = BloomFilterSerializersModule.bloomKFilterFromBytes(decoded);\n} catch (IOException | IllegalArgumentException e) {\n  throw new ISE(\"Bloom filter literal is not valid BloomKFilter bytes: %s\", e.getMessage());\n}","preventionTips":["Generate filter strings only via Druid's ObjectMapper with BloomFilterSerializersModule registered.","Base64-encode/decode round-trip test the filter string before embedding it in SQL.","Avoid passing filter strings through tools that reflow or truncate whitespace."],"tags":["bloom-filter","deserialization","sql"],"backgroundTag":"json-unmarshal-failed","analyzedSha":"9b90983fd291f26935af934383ce360473179e4d","analyzedAt":"2026-09-07T13:32:30.957Z","contentChangedAt":"2026-09-07T13:32:30.957Z","schemaVersion":2},"datasetVersion":"2026-09-17T15:17:12.973Z"}