What causes "sysctl %s has %d keys but only %d defined in f lag" in prometheus/node_exporter?

Count the values with `cat /proc/sys/ ` and supply exactly that many keys

sysctl has keys but only defined in f lag

Error message

sysctl %s has %d keys but only %d defined in f lag

What it means

The sysctl returned `length` values but the user defined a different number of keys in the flag; the mapping from value index to label key is impossible. Note the message's 'f lag' typo is upstream. newMetrics aborts that sysctl's collection.

Solutions

Count the values with `cat /proc/sys/<name>` and supply exactly that many keys
Update the include filter to match the current kernel's layout
Use the indexed form (no keys) to export values by index instead of names

Example fix

// before
--collector.sysctl.include='kernel.sched:cpu1,cpu2'
// after (3 values present)
--collector.sysctl.include='kernel.sched:cpu1,cpu2,cpu3'

Defensive patterns

Strategy: validation

Validate before calling

values := strings.Fields(string(data))
if len(values) != len(keys) { return fmt.Errorf("want %d keys, sysctl has %d", len(keys), len(values)) }

Try / catch

if err != nil && strings.Contains(err.Error(), "keys but only") { /* realign key list with current kernel layout */ }

Prevention

Diff value counts across kernel versions you deploy on
Prefer indexed (key-less) export when key counts are unstable
Keep one include-list config per kernel version

When it happens

Trigger: --collector.sysctl.include='name:k1,k2' where the underlying file has 3 (or 1) values; kernel version changed the value count of a multi-value sysctl.

Common situations: Config written for an older kernel whose multi-value sysctl later gained/lost fields; off-by-one in the comma-separated key list.

Understand the failure class

Background: "Must be a positive integer", "Invalid value", "Unsupported": the invalid-argument-value error family, when a library rejects the value you pass — this error's family across 35 libraries.

Related errors

AI-assisted analysis of prometheus/node_exporter@17ddd77c59 (2026-09-07). Data as JSON: /api/errors/6547a8b13352b2df. Report an issue: GitHub.

Appendix: source

Thrown at collector/sysctl_linux.go:124

	}

	switch length {
	case 0:
		return nil, fmt.Errorf("sysctl %s has no values", s.name)
	case 1:
		if len(s.keys) > 0 {
			return nil, fmt.Errorf("sysctl %s has only one value, but expected %v", s.name, s.keys)
		}
		return []prometheus.Metric{s.newConstMetric(values)}, nil

	default:

		if len(s.keys) == 0 {
			return s.newIndexedMetrics(values), nil
		}

		if length != len(s.keys) {
			return nil, fmt.Errorf("sysctl %s has %d keys but only %d defined in f lag", s.name, length, len(s.keys))
		}

		return s.newMappedMetrics(values)
	}
}

type sysctl struct {
	numeric bool
	name    string
	keys    []string
}

func newSysctl(include string, numeric bool) (*sysctl, error) {
	parts := strings.SplitN(include, ":", 2)
	s := &sysctl{
		numeric: numeric,
		name:    parts[0],
	}

View on GitHub (pinned to 17ddd77c59)