incompatible down-level server detected; servers must be upg

Error message

incompatible down-level server detected; servers must be upgraded to at least %s, or restarted with --disable-network-policy

What it means

validateNetworkConfig runs on the agent and refuses to start the network policy controller when the server-supplied config is incomplete: if DisableNPC is false and either ServiceCIDR is nil or ServiceNodePortRange.Size == 0, the server is declared down-level. Old servers do not send these fields; running the NPC without them would misconfigure network policy, so the agent aborts with the minimum version it needs.

Source

Thrown at pkg/agent/config/config.go:834

	}

	controlControl := &config.Control{}
	return controlControl, json.Unmarshal(data, controlControl)
}

// getReadyz returns nil if the server is ready, or an error if not.
func getReadyz(info *clientaccess.Info) error {
	_, err := info.Get("/v1-" + version.Program + "/readyz")
	return err
}

// validateNetworkConfig ensures that the network configuration values provided by the server make sense.
func validateNetworkConfig(nodeConfig *config.Node) error {
	// Old versions of the server do not send enough information to correctly start the NPC. Users
	// need to upgrade the server to at least the same version as the agent, or disable the NPC
	// cluster-wide.
	if !nodeConfig.AgentConfig.DisableNPC && (nodeConfig.AgentConfig.ServiceCIDR == nil || nodeConfig.AgentConfig.ServiceNodePortRange.Size == 0) {
		return fmt.Errorf("incompatible down-level server detected; servers must be upgraded to at least %s, or restarted with --disable-network-policy", version.Version)
	}

	return nil
}

View on GitHub (pinned to 6ba341e396)

Solutions

Upgrade the server to at least the version printed in the error (the agent's version.Version), then restart it
Or restart the server with --disable-network-policy to run the cluster without the NPC
For rolling upgrades: upgrade servers first, then agents

Defensive patterns

Strategy: validation

Validate before calling

// before enabling the NPC on a new agent, confirm the server sends the needed fields:
// fetch the agent config from the server; if ServiceCIDR == nil or
// ServiceNodePortRange.Size == 0, upgrade the server first or start it with
// --disable-network-policy.

Prevention

Upgrade servers before agents in every rolling upgrade
Pin agent versions to <= server version until the server is upgraded
If agents must outpace the server temporarily, run the server with --disable-network-policy

When it happens

Trigger: An agent at version X talking to a server older than X (server omits serviceCIDR / service-node-port-range from the /v1-<program>/configs payload) while network policy is enabled (server not started with --disable-network-policy).

Common situations: Rolling upgrades where agents were upgraded before servers; a new-version agent joining an old-version server; a downgraded or version-pinned server behind the load balancer.

Related errors

AI-assisted analysis of k3s-io/k3s@6ba341e396 (2026-08-15). Data as JSON: /api/errors/626ef37559d219db. Report an issue: GitHub.