cilium/cilium · error

failed to write update event to WAL: %w

Error message

failed to write update event to WAL: %w

What it means

Update in the route reconciler persists an update event to a write-ahead log (WAL) before replacing the route via netlink, so a crash between WAL write and route update is recoverable. If wal.Write fails, the update is aborted with this wrapped error.

Source

Thrown at pkg/datapath/linux/route/reconciler/reconciler.go:208

		return nil
	}

	if obj.Device != nil {
		// Verify that the device still exists.
		_, _, found := ops.devices.Get(rxn, tables.DeviceByIndex(obj.Device.Index))
		if !found {
			return errDeviceNotFound
		}
	}

	// Write to the WAL first, then update the route in the kernel.
	// If we crash after writing to the WAL but before updating the route, no harm done.
	err := ops.wal.Write(reconcilerEvent{
		Deleted: false,
		Key:     obj.GetOwnerlessKey(),
	})
	if err != nil {
		return fmt.Errorf("failed to write update event to WAL: %w", err)
	}

	return ops.handle.RouteReplace(desiredRouteToNetlinkRoute(obj))
}

func (ops *ops) Delete(_ context.Context, rxn statedb.ReadTxn, _ statedb.Revision, obj *DesiredRoute) error {
	// If the route is not selected, we do not need to delete it.
	if !obj.selected {
		return nil
	}

	// First delete the route from the kernel, then write to the WAL.
	// If we crash after deleting the route but before writing to the WAL, we will
	// re-try the deletion on restart, which is safe.
	err2 := ops.handle.RouteDel(desiredRouteToNetlinkRoute(obj))

	err := ops.wal.Write(reconcilerEvent{
		Deleted: true,

View on GitHub (pinned to ac7b90affa)

Solutions

1. Check disk space and filesystem writability for the WAL directory
2. Fix permissions on the state/WAL directory
3. Inspect the wrapped cause for the exact OS error
4. Move the state directory to a healthy volume and restart

// WAL dir writable?
if err := os.MkdirAll(stateDir, 0o750); err != nil { return err }
probe := filepath.Join(stateDir, ".wal-probe")
if err := os.WriteFile(probe, nil, 0o600); err != nil { return fmt.Errorf("state dir not writable: %w", err) }
os.Remove(probe)

var pathErr *os.PathError
if errors.As(err, &pathErr) { /* inspect pathErr.Err: ENOSPC, EACCES, EROFS */ }

if err := mgr.Update(ctx, txn, rev, route); err != nil {
    if strings.Contains(err.Error(), "failed to write update event to WAL") { /* check disk/permissions on state dir */ }
    return err
}

When it happens

Trigger: ops.wal.Write returns an error during DesiredRoute upsert — typically WAL file open/write failure (disk full, permission denied, directory missing).

Common situations: Full or read-only filesystem hosting the WAL; incorrect state directory permissions; WAL file corrupted or locked by another process.

Related errors

• device no longer exists
• route %s not found
• timeout waiting for parameter %s reconciliation: %w
• timeout waiting for parameter %s reconciliation
• failed to write delete event to WAL: %w

AI-assisted analysis of cilium/cilium@ac7b90affa (2026-08-31). Data as JSON: /api/errors/8eade8adab375253. Report an issue: GitHub.