apache/druid · warning

Expected 0 resource count, got

Error message

Expected 0 resource count, got [%d]! Object was[%s].

What it means

MemcacheClientPool tracks active memcached client leases with a reference count. A watchdog runnable runs at shutdown/pool cleanup and expects the count to be zero; if not, a client was never returned to the pool (leaked). It increments a LEAKED_CLIENTS counter and logs the offending client factory — a diagnostic for a resource leak, not a thrown exception.

Solutions

1. Ensure every client acquisition is paired with a release in try/finally in any custom cache code.
2. Check whether shutdown timing races with active queries; drain queries before shutdown.
3. Upgrade Druid if the leak originates in known pool-lifecycle bugs in older versions.
4. Watch the leaked-clients metric; if it grows continuously, restart brokers to reclaim clients and investigate hold durations.

Example fix

// before
MemcachedClientIF client = pool.get();
client.set(key, payload);
// after
MemcachedClientIF client = pool.get();
try {
  client.set(key, payload);
} finally {
  pool.checkIn(client);
}

try { MemcachedClientIF c = pool.get(); use(c); } finally { pool.checkIn(c); }

When it happens

Trigger: The pool's checker run() observes count.get() != 0, meaning some code path acquired a memcached client from the pool and did not release it (missing returnClient/close, exception bypassing release).

Common situations: Exception thrown between client acquisition and release without try/finally; broker shutdown while cache population in flight; long-running queries holding clients beyond the check interval; Druid version bugs in cache pool lifecycle.

Understand the failure class

Background: 'Could not be found', 'does not exist', 'not found in database': the resource-not-found family when an ID, slug, key, or URI lookup comes back empty — this error's family across 20 libraries.

Related errors

• Could not populate cache
• Exception pulling item from cache
• Invalid value provided for `druid.cache.clientMode`. Value…
• Multiple monitors on the same cache causing race conditions…
• Unable to get SHA1 digest instance

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/3878a018a0b0616a. Report an issue: GitHub.

Appendix: source

Thrown at server/src/main/java/org/apache/druid/client/cache/MemcacheClientPool.java:157

  private static class ClientLeakNotifier implements Runnable
  {
    private final AtomicInteger count;
    private final MemcachedClientIF clientIF;

    private ClientLeakNotifier(AtomicInteger count, MemcachedClientIF clientIF)
    {
      this.count = count;
      this.clientIF = clientIF;
    }

    @Override
    public void run()
    {
      final int shouldBeZero = count.get();
      if (shouldBeZero != 0) {
        LEAKED_CLIENTS.incrementAndGet();
        log.warn("Expected 0 resource count, got [%d]! Object was[%s].", shouldBeZero, clientIF);
      }
    }
  }
}

View on GitHub (pinned to 9b90983fd2)