prometheus/node_exporter · error

couldn't get system state

Error message

couldn't get system state: %w

What it means

This error is returned by systemdCollector.collectSystemState when calling the D-Bus manager property GetManagerProperty("SystemState") fails. systemd's SystemState property reports the overall manager state (e.g. "running", "degraded", "maintenance"); if the property cannot be read, the node_systemd_system_running gauge cannot be computed. The underlying D-Bus error is wrapped with %w so it can be inspected via errors.Is/As.

Solutions

  1. Verify node_exporter can talk to the system bus: check /run/dbus/system_bus_socket exists and is accessible to the exporter user
  2. Restart node_exporter so it re-establishes the D-Bus connection after a systemd restart/upgrade
  3. If running in a container, mount the host D-Bus socket (e.g. -v /run/dbus/system_bus_socket:/run/dbus/system_bus_socket) and run with --collector.systemd
  4. Check journalctl for systemd/polkit errors at scrape time and ensure polkit policy allows org.freedesktop.systemd1.Manager property reads

Example fix

// before: container without dbus socket
docker run prom/node-exporter --collector.systemd
// after
docker run -v /run/dbus/system_bus_socket:/run/dbus/system_bus_socket prom/node-exporter --collector.systemd
Defensive patterns

Strategy: try-catch

Validate before calling

if _, err := os.Stat("/run/dbus/system_bus_socket"); err != nil {
    // systemd collector will fail: dbus socket unavailable
}

Type guard

func dbusAvailable() bool {
    _, err := os.Stat("/run/dbus/system_bus_socket")
    return err == nil
}

Try / catch

if err := collector.Update(ch); err != nil {
    var derr *dbus.Error
    if errors.As(err, &derr) {
        // handle dbus-level failure (reconnect, skip scrape)
    }
    log.Warn("systemd state unavailable", "err", err)
}

Prevention

When it happens

Trigger: The systemd D-Bus manager connection is dead or was closed; the D-Bus socket is unavailable; polkit denies the property read; systemd is being upgraded/restarted during scrape; or the connection was created without the required system bus address.

Common situations: Running node_exporter in a container without mounting /run/dbus/system_bus_socket; systemd restarting mid-scrape during package upgrades; missing systemd user session privileges in restricted environments; DBUS_SYSTEM_BUS_ADDRESS misconfigured.

Understand the failure class

Background: ECONNREFUSED and "connection refused" / "could not connect to server" errors: what they mean and how to fix them — this error's family across 44 libraries.

Related errors

AI-assisted analysis of prometheus/node_exporter@17ddd77c59 (2026-09-07). Data as JSON: /api/errors/39433bf0c7f27202. Report an issue: GitHub.

Appendix: source

Thrown at collector/systemd_linux.go:428

		}

		ch <- prometheus.MustNewConstMetric(
			c.timerLastTriggerDesc, prometheus.GaugeValue,
			float64(lastTriggerValue.Value.Value().(uint64))/1e6, unit.Name)
	}
}

func (c *systemdCollector) collectSummaryMetrics(ch chan<- prometheus.Metric, summary map[string]float64) {
	for stateName, count := range summary {
		ch <- prometheus.MustNewConstMetric(
			c.summaryDesc, prometheus.GaugeValue, count, stateName)
	}
}

func (c *systemdCollector) collectSystemState(conn *dbus.Conn, ch chan<- prometheus.Metric) error {
	systemState, err := conn.GetManagerProperty("SystemState")
	if err != nil {
		return fmt.Errorf("couldn't get system state: %w", err)
	}
	isSystemRunning := 0.0
	if systemState == `"running"` {
		isSystemRunning = 1.0
	}
	ch <- prometheus.MustNewConstMetric(c.systemRunningDesc, prometheus.GaugeValue, isSystemRunning)
	return nil
}

func newSystemdDbusConn() (*dbus.Conn, error) {
	if *systemdPrivate {
		return dbus.NewSystemdConnectionContext(context.TODO())
	}
	return dbus.NewWithContext(context.TODO())
}

type unit struct {
	dbus.UnitStatus

View on GitHub (pinned to 17ddd77c59)