What causes "waiting for endpoint restorer: %w" in cilium/cilium?

Check agent logs for an earlier root-cause failure in the endpoint restorer's startup and fix that first.

waiting for endpoint restorer: %w

Error message

waiting for endpoint restorer: %w

What it means

The per-endpoint-route-initializer job failed while waiting for the endpoint restorer to become available via p.EPRestorer.Await(ctx). The restorer dependency never resolved, most commonly because the context was cancelled during agent shutdown or the restorer's own startup failed / is not wired into the hive.

Source

Thrown at pkg/datapath/loader/endpoint.go:290

	return nil
}

func registerRouteInitializer(p Params) {
	// [upsertEndpointRoute] Creates routes for endpoints that need per endpoint routes.
	// We need to tell the route reconciler to delay pruning of routes from the kernel until we have had a chance
	// to insert desired routes for all endpoints that need them.
	//
	// Use the endpoint restorer to get a signal when all existing endpoints have been restored, and thus
	// [loader.ReloadDatapath] has been called for all existing endpoints. After that we can finalize the route
	// initializer.
	routeInitializer := p.RouteManager.RegisterInitializer("per-endpoint-routes")
	p.JobGroup.Add(job.OneShot("per-endpoint-route-initializer", func(ctx context.Context, _ cell.Health) error {
		defer p.RouteManager.FinalizeInitializer(routeInitializer)

		epRestorer, err := p.EPRestorer.Await(ctx)
		if err != nil {
			return fmt.Errorf("waiting for endpoint restorer: %w", err)
		}

		if err := epRestorer.WaitForEndpointRestore(ctx); err != nil {
			return fmt.Errorf("waiting for endpoint restore: %w", err)
		}

		return nil
	}))
}

func upsertEndpointRoute(db *statedb.DB, devices statedb.Table[*tables.Device], rm *routeReconciler.DesiredRouteManager, ep endpoint.Endpoint, ip netip.Prefix) error {
	owner, err := rm.GetOrRegisterOwner("endpoint/" + ep.StringID())
	if err != nil {
		return fmt.Errorf("getting or registering owner for endpoint %s: %w", ep.StringID(), err)
	}

	// This timeout is 50 times the current batch interval of the devices controller, and thus should
	// be sufficient.

View on GitHub (pinned to ac7b90affa)

Solutions

Check agent logs for an earlier root-cause failure in the endpoint restorer's startup and fix that first.
If the agent was shutting down, this is expected — restart the agent and let startup complete.
Verify the EndpointRestorer cell is enabled (not disabled by flags/config) for your Cilium version.
If startup deadlocks repeatedly, collect a hive dependency dump and check module ordering.

Example fix

// before: await without shutdown awareness
epRestorer, err := p.EPRestorer.Await(ctx)
if err != nil {
    return fmt.Errorf("waiting for endpoint restorer: %w", err)
}
// after: distinguish shutdown from real failure
epRestorer, err := p.EPRestorer.Await(ctx)
if err != nil {
    if ctx.Err() != nil {
        return nil // shutting down; not an error
    }
    return fmt.Errorf("waiting for endpoint restorer: %w", err)
}

Defensive patterns

Strategy: try-catch

Validate before calling

if p.EPRestorer == nil {
    return errors.New("EndpointRestorer cell missing from hive config")
}

Prevention

Keep the EndpointRestorer cell enabled in the hive module graph.
Avoid killing the agent during startup; drain gracefully.
Check earlier logs for restorer startup failures as the root cause.
Re-verify module wiring when upgrading Cilium versions.

When it happens

Trigger: EPRestorer.Await(ctx) returns an error: ctx cancelled (agent shutting down during startup), the EndpointRestorer cell is missing/disabled in the module graph, or the restorer's startup failed so its promise never resolves.

Common situations: Agent killed mid-startup (helm upgrade, OOMKill) before the restorer resolved; cell wiring changed after a Cilium version upgrade; startup dependency deadlock.

Related errors

AI-assisted analysis of cilium/cilium@ac7b90affa (2026-08-31). Data as JSON: /api/errors/8300fb5ded2e19eb. Report an issue: GitHub.