cilium/cilium · error

failed to write cluster configuration: %w

Error message

failed to write cluster configuration: %w

What it means

The enforce job periodically and on-trigger writes the marshalled CiliumClusterConfig to the kvstore key cilium/config/<cluster-name> via Client.UpdateIfDifferent. If that kvstore write fails, this error wraps the underlying cause (context deadline of 5s, kvstore unavailability, permission/lease errors, etc.).

Source

Thrown at pkg/clustermesh/clustercfg/enforce.go:60

	}

	watching := make(chan struct{})
	enforce := func(ctx context.Context) error {
		ctx, cancel := context.WithTimeout(ctx, 5*time.Second)
		defer cancel()

		select {
		// Make sure that the watcher actually started. This is mostly for testing
		// purposes, to prevent possible race conditions, but it is also helpful
		// as a sanity check in production environments.
		case <-watching:
		case <-ctx.Done():
			return fmt.Errorf("timed out waiting for cluster configuration watcher to be started")
		}

		_, err := in.Client.UpdateIfDifferent(ctx, key, value, true)
		if err != nil {
			return fmt.Errorf("failed to write cluster configuration: %w", err)
		}

		return nil
	}

	trigger := job.NewTrigger()
	store := in.StoreFactory.NewWatchStore(
		in.ClusterInfo.Name, store.KVPairCreator,
		&observer{
			key:      in.ClusterInfo.Name,
			expected: value,
			trigger:  trigger,
		},
		store.RWSWithOnSyncCallback(func(context.Context) { close(watching) }),
	)

	in.JobGroup.Add(
		job.OneShot(

View on GitHub (pinned to ac7b90affa)

Solutions

1. Check etcd health and logs; restore kvstore connectivity
2. Look at the wrapped cause in the error to distinguish timeout vs access vs session errors
3. Raise kvstore timeout/resources if the 5s deadline is consistently exceeded under load
4. Verify etcd user permissions allow writes to the cilium/config prefix

// pre-check kvstore write access
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
_, err := kvstoreClient.UpdateIfDifferent(ctx, key, value, true)

if _, err := in.Client.UpdateIfDifferent(ctx, key, value, true); err != nil {
	var retriable bool
	if errors.Is(err, context.DeadlineExceeded) { retriable = true }
	// otherwise inspect wrapped cause: etcd down, permissions, lease lost
}

When it happens

Trigger: UpdateIfDifferent on the cluster-config key fails: kvstore (etcd) unreachable, 5-second context timeout exceeded, key access denied, or kvstore session/lease lost mid-write.

Common situations: etcd outage or network partition during clustermesh operation; etcd compaction/quota exceeded; slow etcd responses making the 5s deadline too tight; RBAC restrictions on the config prefix.

Related errors

• deleting node from kvstore: %w
• timed out waiting for cluster configuration watcher to be st
• %w: %w
• ErrClusterIDChanged
• adding user '%s': %w

AI-assisted analysis of cilium/cilium@ac7b90affa (2026-08-31). Data as JSON: /api/errors/e8bf9161546e276f. Report an issue: GitHub.