cilium/cilium · critical

failed to ensure local routing rule: %w

Error message

failed to ensure local routing rule: %w

What it means

On cell start (when not in dry mode), the proxy lifecycle hook runs linuxdatapath.NodeEnsureLocalRoutingRule to install the local routing rule that steers proxy return traffic. Failure to program this Linux routing rule aborts startup with this wrapped error.

Source

Thrown at pkg/proxy/cell.go:97

	p, err := createProxy(option.Config.EnableL7Proxy, params.Logger, params.LocalNodeStore, params.ProxyPorts, params.EnvoyProxyIntegration, params.DNSProxyIntegration, params.DB, params.Devices, params.RouteManager)
	if err != nil {
		return nil, fmt.Errorf("unable to create proxy: %w", err)
	}

	if !option.Config.EnableL7Proxy {
		params.Logger.Info("L7 proxies are disabled")
		if option.Config.EnableEnvoyConfig {
			params.Logger.Warn("CiliumEnvoyConfig functionality isn't enabled when L7 proxies are disabled", logfields.Flag, option.EnableEnvoyConfig)
		}

		return p, nil
	}

	if !params.DaemonConfig.DryMode {
		params.Lifecycle.Append(cell.Hook{
			OnStart: func(cell.HookContext) error {
				if err := linuxdatapath.NodeEnsureLocalRoutingRule(); err != nil {
					return fmt.Errorf("failed to ensure local routing rule: %w", err)
				}
				return nil
			},
		})
	}

	p.proxyPorts.Trigger = job.NewTrigger(job.WithDebounce(10 * time.Second))

	params.JobGroup.Add(job.OneShot("proxy-ports-restore", func(ctx context.Context, health cell.Health) error {
		if err := p.proxyPorts.RestoreProxyPorts(ctx, health); err != nil {
			// report error to health but proceed to start the checkpoint job
			health.Degraded("restore from file failed", err)
		}

		// Restore all proxy ports before we register the job to overwrite the file below
		params.JobGroup.Add(job.Timer("proxy-ports-checkpoint",
			p.proxyPorts.StoreProxyPorts,
			time.Minute, /* periodic save in case of I/O errors */

View on GitHub (pinned to ac7b90affa)

Solutions

1. Grant the agent CAP_NET_ADMIN and ensure it runs with sufficient privileges (privileged container / host network namespace).
2. Inspect existing 'ip rule' entries and remove stale/conflicting rules at the Cilium priority.
3. Verify kernel/netlink support for policy routing on the host.
4. Use --dry-mode=true for test environments where datapath programming should be skipped.

Example fix

// before: agent without NET_ADMIN fails rule install
docker run cilium/cilium-agent   # missing --cap-add NET_ADMIN

// after
docker run --cap-add NET_ADMIN cilium/cilium-agent

// Pre-flight on the node: can we add policy routing rules?
# ip rule add priority 100 lookup local && ip rule del priority 100 || \
#   echo "cannot manipulate ip rules — missing CAP_NET_ADMIN or conflicting rules"
ip rule show | grep -c 100  # detect pre-existing rule at Cilium's priority

hook := func(cell.HookContext) error {
    if err := linuxdatapath.NodeEnsureLocalRoutingRule(); err != nil {
        if errors.Is(err, os.ErrPermission) {
            return fmt.Errorf("grant NET_ADMIN to the agent: %w", err)
        }
        return fmt.Errorf("failed to ensure local routing rule: %w", err)
    }
    return nil
}

When it happens

Trigger: Agent start (non-dry-mode) where NodeEnsureLocalRoutingRule fails — typically because netlink rules cannot be added: missing CAP_NET_ADMIN, an existing conflicting ip rule with the same priority, or a kernel without rule support.

Common situations: Running cilium-agent in a constrained container without NET_ADMIN; hardened hosts blocking ip rule manipulation; leftover stale routing rules from a previous crashed run.

Related errors

• failed to delete all direct routes: %w
• failed to remove IP %s: %w
• unable to lookup route for node %s: %w
• failed to list direct routes %s: %w
• failed to delete direct route %q: %w

AI-assisted analysis of cilium/cilium@ac7b90affa (2026-08-31). Data as JSON: /api/errors/786f631ab7374c46. Report an issue: GitHub.