prometheus/node_exporter · error

failed to retrieve conntrack stats

Error message

failed to retrieve conntrack stats: %w

What it means

The conntrack collector wraps any error from reading conntrack statistics (/proc/sys/net/netfilter/conntrack_count etc. via procfs) with this message. It distinguishes os.ErrNotExist, which is downgraded to ErrNoData (collector silently reports nothing) since conntrack simply not being loaded is normal, from all other failures which are reported as real errors.

Solutions

1. Mount /proc (read-only) into the container if running containerized
2. Ensure the exporter process can read /proc/sys/net/netfilter/* (check permissions/LSM policy)
3. Load the nf_conntrack module if you expect conntrack metrics, or explicitly disable the conntrack collector
4. Verify --path.procfs points at a valid procfs mount

Example fix

// before
node_exporter --collector.conntrack   # fails in container without /proc/sys/net/netfilter
// after
docker run -v /proc:/host/proc:ro node_exporter --path.procfs=/host/proc --collector.conntrack

// Go: pre-check the proc files exist before scraping
if _, err := os.Stat(filepath.Join(*procPath, "sys/net/netfilter")); os.IsNotExist(err) {
    return ErrNoData // skip conntrack collection
}

func conntrackAvailable(procPath string) bool {
    _, err := os.Stat(filepath.Join(procPath, "sys/net/netfilter"))
    return !os.IsNotExist(err)
}

stats, err := getConntrackStatistics()
if err != nil {
    if errors.Is(err, ErrNoData) || errors.Is(err, os.ErrNotExist) {
        return nil // treat as absent feature
    }
    return fmt.Errorf("conntrack: %w", err)
}

When it happens

Trigger: Update() -> getConntrackStatistics() fails reading /proc/sys/net/netfilter/ files for a reason other than file-not-exist, e.g. permission problems or procfs parse errors.

Common situations: Containerized exporters without the netfilter proc files mounted; hardened systems where the exporter user cannot read /proc/sys/net/netfilter; kernel without nf_conntrack exposing partial files; procPath pointed at a non-Linux/odd /proc tree.

Understand the failure class

Background: "failed to read file", EACCES, ENOENT and "could not read <path>" errors: when a program can't read a file from disk — this error's family across 49 libraries.

Related errors

• interrupts empty
• ZFS / ZFS statistics are not available
• failed to open procfs
• failed to open procfs
• couldn't get buddyinfo

AI-assisted analysis of prometheus/node_exporter@17ddd77c59 (2026-09-07). Data as JSON: /api/errors/157c393f7ab70506. Report an issue: GitHub.

Appendix: source

Thrown at collector/conntrack_linux.go:151

	ch <- prometheus.MustNewConstMetric(
		conntrackInsert, prometheus.GaugeValue, float64(conntrackStats.insert))
	ch <- prometheus.MustNewConstMetric(
		conntrackInsertFailed, prometheus.GaugeValue, float64(conntrackStats.insertFailed))
	ch <- prometheus.MustNewConstMetric(
		conntrackDrop, prometheus.GaugeValue, float64(conntrackStats.drop))
	ch <- prometheus.MustNewConstMetric(
		conntrackEarlyDrop, prometheus.GaugeValue, float64(conntrackStats.earlyDrop))
	ch <- prometheus.MustNewConstMetric(
		conntrackSearchRestart, prometheus.GaugeValue, float64(conntrackStats.searchRestart))
	return nil
}

func (c *conntrackCollector) handleErr(err error) error {
	if errors.Is(err, os.ErrNotExist) {
		c.logger.Debug("conntrack probably not loaded")
		return ErrNoData
	}
	return fmt.Errorf("failed to retrieve conntrack stats: %w", err)
}

func getConntrackStatistics() (*conntrackStatistics, error) {
	s := conntrackStatistics{}

	fs, err := procfs.NewFS(*procPath)
	if err != nil {
		return nil, fmt.Errorf("failed to open procfs: %w", err)
	}

	connStats, err := fs.ConntrackStat()
	if err != nil {
		return nil, err
	}

	for _, connStat := range connStats {
		s.found += connStat.Found
		s.invalid += connStat.Invalid

View on GitHub (pinned to 17ddd77c59)