apache/druid · error · IllegalStateException

There are too many [ ] pendingNotices.

Error message

There are too many [%d] pendingNotices.

What it means

addNotice() caps the pending notice queue at 10000 entries to prevent unbounded growth. If more than 10000 notices are already pending (i.e., the main handling thread is not draining them), adding another add/remove notice throws an IllegalStateException.

Solutions

1. Check why the mainThread notice handler is blocked (thread dump; usually stuck in factory start()/close()).
2. Rate-limit or batch add()/remove() calls instead of issuing them per-key in tight loops.
3. Verify the manager's lifecycle is started so notices are actually being consumed.
4. Wait for the queue to drain before issuing more updates (monitor pendingNotices size).

Example fix

// before: unbounded bulk updates
for (Lookup l : allLookups) { manager.add(l.getName(), l.getContainer()); }
// after: throttle and wait for drain
while (pendingNoticesSize(manager) > 9000) { Thread.sleep(100); }
for (Lookup l : allLookups) { manager.add(l.getName(), l.getContainer()); }

// before bulk updates, wait until the notice queue is drained
while (pendingNoticeCount(manager) > 9000) { Thread.sleep(100); }

try { manager.add(name, container); } catch (IllegalStateException e) { if (e.getMessage().contains("pendingNotices")) { backoffAndRetry(name, container); } else { throw e; } }

When it happens

Trigger: Calling add() or remove() (directly or via lookup coordinator updates) more than ~10000 times while the background mainThread is stalled, not started, or slower than notice production.

Common situations: Bulk lookup sync loops pushing thousands of add/remove calls; main lookup-update thread hung on a slow lookup factory start(); leaked or blocked mainThread after lifecycle problems.

Related errors

• failed to stop, mainThread couldn't finish.
• LookupCoordinatorManager executor from last start() hasn't…
• OpenLineage emit queue full, discarded [%,d] events total
• Already closed
• Already closed

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/3987df2a8f372cf5. Report an issue: GitHub.

Appendix: source

Thrown at server/src/main/java/org/apache/druid/query/lookup/LookupReferencesManager.java:298

          lookupLoadingSpec.getMode()
      );
      return;
    }
    addNotice(new LoadNotice(lookupName, lookupExtractorFactoryContainer, lookupConfig.getLookupStartRetries()));
  }

  public void remove(String lookupName, LookupExtractorFactoryContainer loadedContainer)
  {
    Preconditions.checkState(lifecycleLock.awaitStarted(1, TimeUnit.MILLISECONDS));
    addNotice(new DropNotice(lookupName, loadedContainer));
  }

  private void addNotice(Notice notice)
  {
    atomicallyUpdateStateRef(
        oldState -> {
          if (oldState.pendingNotices.size() > 10000) { //don't let pendingNotices grow indefinitely
            throw new ISE("There are too many [%d] pendingNotices.", oldState.pendingNotices.size());
          }

          ImmutableList.Builder<Notice> builder = ImmutableList.builder();
          builder.addAll(oldState.pendingNotices);
          builder.add(notice);

          return new LookupUpdateState(oldState.lookupMap, builder.build(), oldState.noticesBeingHandled);
        }
    );
    LockSupport.unpark(mainThread);
  }

  public void submitAsyncLookupTask(Runnable task)
  {
    lookupUpdateExecutorService.submit(task);
  }

  @Override

View on GitHub (pinned to 9b90983fd2)