prometheus/node_exporter · critical

calloc() failed

Error message

calloc() failed

What it means

In the DragonFly BSD devstat collector (devstat_dragonfly.go), C._get_ndevs() returns -2 when its internal calloc() of the device-stat array fails, i.e. the process could not allocate memory for device statistics. Update returns errors.New("calloc() failed") and this scrape's disk metrics are dropped.

Solutions

1. Check host free memory and rlimits (ulimit -v, jail/cgroup memory caps) and raise them for node_exporter
2. Restart node_exporter if the process is leaking or fragmented
3. Investigate why devstat reports a very large device count; a corrupt devstat state can trigger oversized allocations
4. Disable the devstat collector if memory-constrained hosts cannot afford it

if err := c.Update(ch); err != nil {
    if strings.Contains(err.Error(), "calloc() failed") {
        c.logger.Error("devstat allocation failed; check host memory/rlimits", "err", err)
    }
    return err // non-retryable until memory frees up; alert on it
}

When it happens

Trigger: C._get_ndevs() == -2 in devstatCollector.Update — the C helper's calloc(count, sizeof(...)) returned NULL, which happens when the requested allocation size exceeds available memory.

Common situations: Hosts under severe memory pressure or with very low rlimits (ulimit -v / jail memory caps); pathological device counts reported by devstat causing a huge allocation.

Related errors

• getdevs() failed
• could not retrieve CPU times
• devstat_getdevs failed
• getmntinfo() failed
• getifaddrs() failed

AI-assisted analysis of prometheus/node_exporter@17ddd77c59 (2026-09-07). Data as JSON: /api/errors/4a38d14cf7baa296. Report an issue: GitHub.

Appendix: source

Thrown at collector/devstat_dragonfly.go:135

			"The total number of transactions completed.",
			[]string{"device"}, nil,
		),
		blocksDesc: prometheus.NewDesc(
			prometheus.BuildFQName(namespace, devstatSubsystem, "blocks_total"),
			"The total number of bytes given in terms of the devices blocksize.",
			[]string{"device"}, nil,
		),
		logger: logger,
	}, nil
}

func (c *devstatCollector) Update(ch chan<- prometheus.Metric) error {
	count := C._get_ndevs()
	if count == -1 {
		return errors.New("getdevs() failed")
	}
	if count == -2 {
		return errors.New("calloc() failed")
	}

	for i := C.int(0); i < count; i++ {
		stats := C._get_stats(i)
		device := fmt.Sprintf("%s%d", C.GoString(&stats.device[0]), stats.unit)

		ch <- prometheus.MustNewConstMetric(c.bytesDesc, prometheus.CounterValue, float64(stats.bytes), device)
		ch <- prometheus.MustNewConstMetric(c.transfersDesc, prometheus.CounterValue, float64(stats.transfers), device)
		ch <- prometheus.MustNewConstMetric(c.blocksDesc, prometheus.CounterValue, float64(stats.blocks), device)
	}

	return nil
}

View on GitHub (pinned to 17ddd77c59)