netdata/netdata · error

metrix: write outside active cycle

Error message

metrix: write outside active cycle

What it means

This panic fires in storeCore.recordStateSetObserve (stateset.go:112) when c.active is nil, i.e. no collection cycle is currently active. The snapshot storeCore buffers writes into per-cycle frames created when a cycle begins; stateset observations arriving before Begin or after the cycle is closed/flushed have no frame to write into, so the write outside an active cycle panics. The same guard exists on gauge, counter, histogram, and summary write paths.

Source

Thrown at src/go/pkg/metrix/stateset.go:112

	for _, st := range schema.states {
		states[st] = false
	}
	for _, active := range actives {
		if _, ok := schema.index[active]; !ok {
			panic(errStateSetUnknownState)
		}
		states[active] = true
	}
	return StateSetPoint{States: states}
}

// recordStateSetObserve writes one full-state stateset sample into the active frame.
func (c *storeCore) recordStateSetObserve(desc *instrumentDescriptor, scope HostScope, point StateSetPoint, sets []LabelSet) {
	c.mu.Lock()
	defer c.mu.Unlock()

	if c.active == nil {
		panic(errCycleInactive)
	}

	schema := desc.stateSet
	if schema == nil {
		panic(errStateSetSchema)
	}

	labels, labelsKey, err := labelsFromSet(sets, c)
	if err != nil {
		panic(err)
	}
	if labelsContainKey(labels, desc.name) {
		panic(errStateSetLabelKey)
	}
	scope, ok := c.prepareHostScopeForWriteLocked(scope)
	if !ok {
		return
	}

View on GitHub (pinned to 4864de85e2)

Solutions

  1. Move the write inside the active collection cycle (inside the Collect function between cycle begin and end).
  2. If writes originate in async callbacks, buffer them in a channel/queue and drain them during the next cycle instead of writing directly.
  3. In tests, wrap writes with the store's cycle begin/end helpers.

Example fix

// before
go func() {
    ss.ObserveStateSet(p) // fires after cycle end -> panic
}()

// after
ch <- p                 // async producer
// inside Collect():
for p := range drain(ch) {
    ss.ObserveStateSet(p)
}
Defensive patterns

Strategy: validation

Validate before calling

// Buffer async producers and drain inside the cycle:
// producer (any time):   events <- p
// inside Collect():      for p := range drain(events) { ss.ObserveStateSet(p) }

Prevention

When it happens

Trigger: Calling ObserveStateSet from a goroutine or callback that runs outside the collector's Collect/scrape window — e.g. an event handler or background updater firing after the cycle ended, or writes issued before the store's cycle was started.

Common situations: Spawning goroutines that record metrics asynchronously while the scrape cycle advances; collectors that emit from timers or SNMP trap callbacks not synchronized with the collection loop; test code writing to the store without beginning a cycle.

Related errors


AI-assisted analysis of netdata/netdata@4864de85e2 (2026-08-15). Data as JSON: /api/errors/069faa6617b18a87. Report an issue: GitHub.