cilium/cilium · error

failed to delete direct route %q: %w

Error message

failed to delete direct route %q: %w

What it means

Thrown in deleteDirectRoute when netlink.RouteDel fails to remove one specific direct node route after it was successfully listed. Individual failures are logged as warnings and aggregated via errors.Join, so a single bad route does not stop deletion of the others; the joined aggregate is wrapped by the caller. This leaves the failing route installed on the host.

Source

Thrown at pkg/datapath/linux/node.go:341

		Dst:      netipx.PrefixIPNet(prefix),
		Gw:       nodeIP,
		Protocol: linux_defaults.RTProto,
	}

	routes, err := safenetlink.RouteListFiltered(family, filter, netlink.RT_FILTER_DST|netlink.RT_FILTER_GW)
	if err != nil {
		n.log.Error("Unable to list direct routes", logfields.Error, err)
		return fmt.Errorf("failed to list direct routes %s: %w", familyStr, err)
	}

	var errs error
	for _, rt := range routes {
		if err := netlink.RouteDel(&rt); err != nil {
			n.log.Warn("Unable to delete direct node route",
				logfields.CIDR, rt,
				logfields.Error, err,
			)
			errs = errors.Join(errs, fmt.Errorf("failed to delete direct route %q: %w", rt.String(), err))
		}
	}
	return errs
}

// createNodeRouteSpec creates a route spec that points the specified prefix to the host
// device via the router IP. The route is configured with a computed MTU for non-local
// nodes (i.e isLocalNode is set to false).
//
// Example:
// 10.10.0.0/24 via 10.10.0.1 dev cilium_host src 10.10.0.1
// f00d::a0a:0:0:0/112 via f00d::a0a:0:0:1 dev cilium_host src fd04::11 metric 1024 pref medium
func (n *linuxNodeHandler) createNodeRouteSpec(prefix netip.Prefix, isLocalNode bool) (route.Route, error) {
	var (
		local   net.IP
		nexthop *net.IP
		mtu     int
	)

View on GitHub (pinned to ac7b90affa)

Solutions

1. Re-run node update / restart cilium-agent: this error is often transient (route vanished mid-scan) and the next reconcile clears it.
2. Ensure CAP_NET_ADMIN is granted to the agent process for RouteDel.
3. Check for conflicting route managers (NetworkManager, dhclient, other CNIs) mutating the same routes and disable the conflict.
4. Use `ip route del` manually for the specific %q route printed in the error, then verify agent reconciliation succeeds.

Example fix

// before: treating the aggregate as fatal and crashing
if err := n.deleteAllDirectRoutes(removedCIDRs, oldIP); err != nil {
    return fmt.Errorf("failed to delete all direct routes: %w", err)
}
// after: log and continue, letting periodic reconciliation retry
if err := n.deleteAllDirectRoutes(removedCIDRs, oldIP); err != nil {
    n.log.Warn("some direct routes could not be deleted, will retry on next reconcile",
        logfields.Error, err)
}

// preflight: confirm the route still exists right before deleting it
routes, err := safenetlink.RouteListFiltered(family, filter, netlink.RT_FILTER_DST|netlink.RT_FILTER_GW)
if err != nil || len(routes) == 0 {
    return nil // nothing to delete; avoids ESRCH/EINVAL on stale routes
}

if err := netlink.RouteDel(&rt); err != nil {
    var errno syscall.Errno
    if errors.As(err, &errno) && (errno == syscall.ESRCH || errno == syscall.ENODEV) {
        // route already gone or device removed — treat as success, continue
        continue
    }
    errs = errors.Join(errs, err) // real failure: aggregate and retry later
}

When it happens

Trigger: During deleteAllDirectRoutes (nodeUpdate removed CIDRs) or nodeDelete, netlink.RouteDel returns an error for a listed route: typically EPERM (missing capability), EINVAL (route already changed/removed by another writer), or ENODEV (the route's link index disappeared between listing and deletion).

Common situations: Races with the kernel removing the interface or with other route managers (e.g. static NetworkManager routes, other CNI remnants) deleting or mutating the same route; unprivileged execution; stale routes referencing a deleted device after node restart.

Related errors

• failed to delete all direct routes: %w
• failed to list direct routes %s: %w
• failed to fetch the probe GENEVE device: %w
• failed to ensure local routing rule: %w
• inserting ipv4 proxy route %v: %w

AI-assisted analysis of cilium/cilium@ac7b90affa (2026-08-31). Data as JSON: /api/errors/95e3a0805f40a3cf. Report an issue: GitHub.