apache/druid · warning

Not running cleanup or callbacks for non-existing segment

Error message

Not running cleanup or callbacks for non-existing segment[%s] on server[%s]

What it means

When a sync asks the view to remove a segment that is not present in the server's tracked inventory, the view skips cleanup and segmentRemoved callbacks. In a delta sync this indicates the data server announced a removal for a segment the view never recorded; in a full sync (post-reset) it is expected and silent.

Solutions

  1. Ignore if it occurs right after a fullSync/reset — expected during re-baselining.
  2. If seen on delta syncs, restart the affected data server so its announced inventory matches reality.
  3. Check for duplicate segment drop announcements in the data server logs.
  4. Verify segment cleanup on the data server is not announcing drops twice.
Defensive patterns

Strategy: validation

Validate before calling

if (serverInventory.containsKey(segment.getId())) { view.removeSegment(...); } else { log.debug("Skip unknown segment {}", segment.getId()); }

Prevention

When it happens

Trigger: removeSegment is called from fullSync or deltaSync with a segment id absent from the druidServer's inventory, e.g. the data server's inventory list no longer contains a segment that was never added or was already removed.

Common situations: Dropped or out-of-order delta sync messages; a server was reset (fullSync baseline) while segments were concurrently dropped; duplicate drop announcements from the data server after a failed ack.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors


AI-assisted analysis of apache/druid@9b90983fd2 (2026-09-07). Data as JSON: /api/errors/0a0d5635cd966eac. Report an issue: GitHub.

Appendix: source

Thrown at server/src/main/java/org/apache/druid/client/HttpServerInventoryView.java:715

      return PartialLoadProfile.forLoaded(loadSpec, fingerprint, loadedBytes);
    }

    private void removeSegment(final DataSegment segment, boolean fullSync)
    {
      if (druidServer.removeDataSegment(segment.getId()) != null) {
        runSegmentCallbacks(
            new Function<>()
            {
              @Override
              public CallbackAction apply(SegmentCallback input)
              {
                return input.segmentRemoved(druidServer.getMetadata(), segment);
              }
            }
        );
      } else if (!fullSync) {
        // duplicates can happen when doing a full sync from a 'reset', so only warn for dupes on delta changes
        log.warn(
            "Not running cleanup or callbacks for non-existing segment[%s] on server[%s]",
            segment.getId(),
            druidServer.getName()
        );
      }
    }
  }
}

View on GitHub (pinned to 9b90983fd2)