apache/druid · warning

Received out-of-order update for table

Error message

Received out-of-order update for table: %s. Cache: %d, update:%d

What it means

checkVersion() detects an update whose updateTime is older than the timestamp already cached for the table — i.e. an out-of-order (stale) update notification. The stale update is rejected (the method returns false) and a forced resync is planned (TODO in code). This guards the cache against regressing to older metadata.

Solutions

1. Ignore if rare — the stale update is correctly rejected; the cache keeps the newer version.
2. Check for multiple processes writing to the same catalog table concurrently and ensure a single writer path.
3. Synchronize clocks (NTP) on hosts producing catalog updates to avoid spurious out-of-order detection.
4. Trigger resync() to pull the authoritative state, since the code's TODO force-resync is not yet implemented.

Example fix

// before
// stale notification arrives; cache keeps old view until next periodic resync
// after
receiver.resync(); // explicitly reconcile cache when out-of-order updates are observed

if (update.updateTime() <= entry.table.updateTime()) { skipStaleUpdate(); scheduleResync(); }

When it happens

Trigger: computeUpdate/computeColumnsUpdate/computePropertiesUpdate receives a TableMetadata update whose updateTime() is less than entry.table.updateTime(); concurrent writers to the same table delivering events out of order; message bus reordering or replay.

Common situations: Multiple catalog writers (e.g. two coordinator/broker sessions) updating the same table; clock skew between catalog writer nodes; replayed notifications after reconnect; partitioned delivery channels delivering out of order.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• Received creation event for existing entry
• Attempt to add row to swapped-out sink for segment
• Background lookup manager thread could not be cancelled
• can't start.
• cannot add element of size

AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/ef552b41914ca638. Report an issue: GitHub.

Appendix: source

Thrown at extensions-core/druid-catalog/src/main/java/org/apache/druid/catalog/sync/CachedMetadataCatalog.java:258

    }

    private boolean checkResolved(TableEntry entry, TableMetadata update, String action)
    {
      if (entry.table == null) {
        LOG.error("Received %s update for unresolved table: %s",
            action,
            update.id().sqlName()
        );
        // TODO: force resync
        return false;
      }
      return true;
    }

    private boolean checkVersion(TableEntry entry, TableMetadata update)
    {
      if (entry.table.updateTime() > update.updateTime()) {
        LOG.warn(
            "Received out-of-order update for table: %s. Cache: %d, update:%d",
            update.id().sqlName(),
            entry.table.updateTime(),
            update.updateTime()
        );
        // TODO: force resync
        return false;
      }
      return true;
    }

    public synchronized Set<String> tableNames()
    {
      Set<String> tables = new HashSet<>();
      cache.forEach((k, v) -> {
        if (v.table != null) {
          tables.add(k);
        }

View on GitHub (pinned to 9b90983fd2)