{"record":{"id":"964d73d186f15560","repo":"hashicorp/nomad","slug":"event-dropped-from-buffer","errorCode":null,"errorMessage":"event dropped from buffer","messagePattern":"event dropped from buffer","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"warning","filePath":"nomad/stream/event_buffer.go","lineNumber":273,"sourceCode":"// Next return the next buffer item in the buffer. It may block until ctx is\n// cancelled or until the next item is published.\nfunc (i *bufferItem) Next(ctx context.Context, forceClose <-chan struct{}) (*bufferItem, error) {\n\t// See if there is already a next value, block if so. Note we don't rely on\n\t// state change (chan nil) as that's not threadsafe but detecting close is.\n\tselect {\n\tcase <-ctx.Done():\n\t\treturn nil, ctx.Err()\n\tcase <-forceClose:\n\t\treturn nil, fmt.Errorf(\"subscription closed\")\n\tcase <-i.link.nextCh:\n\t}\n\n\t// Check if the reader is too slow and the event buffer as discarded the event\n\t// This must happen after the above select to prevent a random selection\n\t// between linkCh and droppedCh\n\tselect {\n\tcase <-i.link.droppedCh:\n\t\treturn nil, fmt.Errorf(\"event dropped from buffer\")\n\tdefault:\n\t}\n\n\t// If channel closed, there must be a next item to read\n\tnextRaw := i.link.next.Load()\n\tif nextRaw == nil {\n\t\t// shouldn't be possible\n\t\treturn nil, errors.New(\"invalid next item\")\n\t}\n\tnext := nextRaw.(*bufferItem)\n\tif next.Err != nil {\n\t\treturn nil, next.Err\n\t}\n\treturn next, nil\n}\n\n// NextNoBlock returns the next item in the buffer without blocking. If it\n// reaches the most recent item it will return nil.","sourceCodeStart":255,"sourceCodeEnd":291,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/nomad/stream/event_buffer.go#L255-L291","documentation":"Next() checks link.droppedCh right after waking on nextCh: if the buffer advanced past events the reader never consumed (reader too slow), it returns \"event dropped from buffer\". This is a liveness signal — the subscription stays valid, but a gap exists between the last delivered event and the next one, so consumers must not assume contiguous state.","triggerScenarios":"Consuming events slower than the producer publishes; the ring buffer wraps and drops events before Next() reads them, firing droppedCh, on any active event stream subscription (job/evaluation/deployment topics).","commonSituations":"UIs or controllers processing heavy event bursts (many concurrent deployments) with per-event work or slow RPCs; long blocking handlers between Next() calls; under-provisioned clients on busy clusters.","solutions":["On this error, re-fetch the current state of the tracked resources and continue consuming (resync-on-gap pattern) — do not treat the stream as dead.","Make per-event handling faster/offloaded so Next() is called promptly.","Reduce consumed topic set or filter events server-side to lower throughput.","Increase buffer capacity if you control the server configuration."],"exampleFix":"// before\nif err.Error() == \"event dropped from buffer\" { return err }\n// after\nif err.Error() == \"event dropped from buffer\" {\n  if err := resyncState(); err != nil { return err } // gap detected: rebuild from API\n  continue // subscription still valid\n}","handlingStrategy":"fallback","validationCode":null,"typeGuard":"func isEventDropped(err error) bool { return err != nil && err.Error() == \"event dropped from buffer\" }","tryCatchPattern":"ev, err := sub.Next(ctx)\nif isEventDropped(err) {\n  if rerr := resyncTrackedState(); rerr != nil { return rerr }\n  continue // subscription remains usable\n}","preventionTips":["Make per-event processing fast; never block long between Next() calls.","Implement resync-on-gap so dropped events never cause silent divergence.","Filter topics/subjects to reduce event throughput to what the consumer can sustain."],"tags":["nomad","event-stream","backpressure","streaming"],"backgroundTag":"event-buffer-lag","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}