apache/druid · error · RuntimeException

interrupted flushing elements from queue

Error message

interrupted flushing elements from queue

What it means

deserialize reads a header byte (numHashFunc), an int (bitset length), and then that many longs from the buffer. Any RuntimeException while reading (BufferUnderflowException from truncated data, IndexOutOfBoundsException from a negative length, etc.) is caught and rethrown as an IOException with message 'Unable to deserialize BloomKFilter' and the original exception as the cause. It indicates the bytes at the given position are not a complete, well-formed BloomKFilter serialization.

Source

Thrown at extensions-contrib/ambari-metrics-emitter/src/main/java/org/apache/druid/emitter/ambari/metrics/AmbariMetricsEmitter.java:266

      catch (Exception e) {
        log.error(e, e.getMessage());
      }

    }
  }

  @Override
  public void flush()
  {
    synchronized (started) {
      if (started.get()) {
        Future future = exec.schedule(new ConsumerRunnable(), 0, TimeUnit.MILLISECONDS);
        try {
          future.get(DEFAULT_FLUSH_TIMEOUT_MILLIS, TimeUnit.MILLISECONDS);
        }
        catch (InterruptedException | ExecutionException | TimeoutException e) {
          if (e instanceof InterruptedException) {
            throw new RuntimeException("interrupted flushing elements from queue", e);
          }
        }
      }
    }
  }

  @Override
  public void close()
  {
    synchronized (started) {
      flush();
      exec.shutdown();
      started.set(false);
    }
  }

  protected static String sanitize(String namespace)
  {

View on GitHub (pinned to 9b90983fd2)

Solutions

  1. Inspect the cause (getCause()) to see whether it is BufferUnderflow (truncated) or a bad length value, and fix the slicing/offset that produced the buffer.
  2. Verify the serialized byte range: store and pass the exact length returned by BloomKFilter.getSerializedSize() for the filter.
  3. Confirm the buffer was serialized with BloomKFilter.serialize by the same (or compatible) Druid version.
  4. Wrap in try-catch for IOException and treat the filter as missing/unknown rather than crashing the query.

Example fix

// before
byte[] slice = new byte[16]; // guessed length, too small
BloomKFilter bf = BloomKFilter.deserialize(ByteBuffer.wrap(slice), 0); // BufferUnderflow -> IOException

// after
byte[] bytes = filterBytes.array();
BloomKFilter bf = BloomKFilter.deserialize(ByteBuffer.wrap(bytes, offset, BloomKFilter.getSerializedSize()), 0);
Defensive patterns

Strategy: validation

Validate before calling

boolean looksLikeBloomFilter(ByteBuffer buf, int position) {
  if (buf == null || position < 0 || buf.remaining() <= position) return false;
  int bitsetLongs = buf.duplicate().order(ByteOrder.BIG_ENDIAN).getInt(position + 1);
  return bitsetLongs >= 0
      && (position + BloomKFilter.START_OF_SERIALIZED_LONGS + (long) bitsetLongs * Long.BYTES) <= buf.capacity();
}

Type guard

boolean hasCompleteFilter(byte[] bytes, int offset) {
  if (bytes == null || offset + 5 > bytes.length) return false;
  int longs = ((bytes[offset+1] & 0xFF) << 24) | ((bytes[offset+2] & 0xFF) << 16)
            | ((bytes[offset+3] & 0xFF) << 8) | (bytes[offset+4] & 0xFF);
  return offset + BloomKFilter.START_OF_SERIALIZED_LONGS + (long) longs * Long.BYTES <= bytes.length;
}

Try / catch

try {
  return BloomKFilter.deserialize(buffer, position);
} catch (IOException e) {
  LOG.warn(e, "Corrupt bloom filter bytes at position %d", position);
  return null; // degrade to 'unknown membership'
}

Prevention

When it happens

Trigger: Calling BloomKFilter.deserialize(ByteBuffer, position) where fewer bytes remain than the header declares (e.g. buffer was truncated, wrong position given, wrong offset/length stored), or the data is corrupt/garbage so the declared bitsetArrayLen is invalid.

Common situations: Storing the serialized filter with a wrong length/offset (e.g. slicing a byte[] incorrectly); passing bytes produced by a different format or Druid version; reading the filter from a partially flushed/failed write; deserializing at a nonzero position into a buffer that does not contain a filter there.

Understand the failure class

Background: "Invalid ... format", "must be in format X", "does not look like a ..." — invalid argument format errors across CLI tools and libraries — this error's family across 17 libraries.

Related errors

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/bdcf362e299328e0. Report an issue: GitHub.