apache/druid · error · IOException

Unable to deserialize BloomKFilter

Error message

Unable to deserialize BloomKFilter

What it means

BloomKFilter.deserialize() reads the header (numHashFunctions, bitset length) and the long[] bitset from the buffer inside a try block; any RuntimeException — most often a negative bitset length, buffer underflow, or invalid hash-function count caused by corrupt or truncated serialized bytes — is caught and rethrown as an IOException with this message. The faulty input is the serialized bloom filter bytes.

Solutions

1. Check whether the serialized bloom filter bytes were truncated or corrupted in storage or transit
2. Verify the filter was written by a compatible BloomKFilter version (header layout matches BIG_ENDIAN expectations)
3. Re-aggregate or re-ingest the affected data to regenerate valid filters

When it happens

Trigger: Thrown at extensions-core/druid-bloom-filter/src/main/java/org/apache/druid/query/filter/BloomKFilter.java:285 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/6c477844d4045374. Report an issue: GitHub.

Appendix: source

Thrown at extensions-core/druid-bloom-filter/src/main/java/org/apache/druid/query/filter/BloomKFilter.java:285

  public static BloomKFilter deserialize(ByteBuffer in, int position) throws IOException
  {
    if (in == null) {
      throw new IOException("Input stream is null");
    }

    try {
      ByteBuffer dataBuffer = in.duplicate().order(ByteOrder.BIG_ENDIAN);
      dataBuffer.position(position);
      int numHashFunc = dataBuffer.get();
      int bitsetArrayLen = dataBuffer.getInt();
      long[] data = new long[bitsetArrayLen];
      for (int i = 0; i < bitsetArrayLen; i++) {
        data[i] = dataBuffer.getLong();
      }
      return new BloomKFilter(data, numHashFunc);
    }
    catch (RuntimeException e) {
      throw new IOException("Unable to deserialize BloomKFilter", e);
    }
  }

  /**
   * Merges BloomKFilter bf2Buffer into bf1Buffer in place. Does not mutate buffer positions.
   * Assumes 2 BloomKFilters with the same size/hash functions are serialized to ByteBuffers
   *
   * @param bf1Buffer
   * @param bf1Start
   * @param bf2Buffer
   * @param bf2Start
   */
  public static void mergeBloomFilterByteBuffers(
      ByteBuffer bf1Buffer,
      int bf1Start,
      ByteBuffer bf2Buffer,
      int bf2Start
  )

View on GitHub (pinned to 9b90983fd2)