What causes "kubernetes configmap delete %s" in go-kratos/kratos?

Recreate the ConfigMap with the same name (kubectl apply) — events resume once it exists again

kubernetes configmap delete %s

Error message

kubernetes configmap delete %s

What it means

Reported when the watched ConfigMap receives a DELETED event: the watcher turns the deletion into an error ('kubernetes configmap delete <name>') to signal that the config source vanished. Deletion usually comes from kubectl, Helm pruning, or garbage collection while the app runs.

Source

Thrown at contrib/config/kubernetes/watcher.go:53

	ch := <-w.watcher.ResultChan()
	if ch.Object == nil {
		// recreate the watcher
		k8sWatcher, err := w.k.client.CoreV1().ConfigMaps(w.k.opts.Namespace).Watch(context.Background(), metav1.ListOptions{
			LabelSelector: w.k.opts.LabelSelector,
			FieldSelector: w.k.opts.FieldSelector,
		})
		if err != nil {
			return nil, err
		}
		w.watcher = k8sWatcher
		goto ResultChan
	}
	cm, ok := ch.Object.(*v1.ConfigMap)
	if !ok {
		return nil, fmt.Errorf("kubernetes Object not ConfigMap")
	}
	if ch.Type == "DELETED" {
		return nil, fmt.Errorf("kubernetes configmap delete %s", cm.Name)
	}
	return w.k.configMap(*cm), nil
}

func (w *watcher) Stop() error {
	w.watcher.Stop()
	return nil
}

View on GitHub (pinned to 668db92c2c)

Solutions

Recreate the ConfigMap with the same name (kubectl apply) — events resume once it exists again
Use a create-before-delete or apply-over strategy to avoid a no-config window
Treat the error as a signal: keep last-known config values, alert, and wait for re-creation instead of crashing

Defensive patterns

Strategy: try-catch

Validate before calling

// pre-check the ConfigMap exists before starting the app
_, err := k8scli.CoreV1().ConfigMaps(ns).Get(ctx, name, metav1.GetOptions{})
if err != nil {
    return fmt.Errorf("configmap %s missing: %w", name, err)
}

Try / catch

if err != nil {
    if strings.HasPrefix(err.Error(), "kubernetes configmap delete") {
        // config removed: keep last-known values, alert, wait for re-create
        alertConfigDeleted()
        return nil
    }
    return err
}

Prevention

Recreate ConfigMaps with kubectl apply, never delete-then-new with a gap
Use labels plus ownerReferences so Helm/GC does not prune config
Alert on deletion events instead of crashing

When it happens

Trigger: The ConfigMap being watched is deleted (kubectl delete, helm upgrade pruning, namespace teardown, GC) so the watch loop receives an event with Type == DELETED.

Common situations: Ops cleanup during debugging; helm upgrade removing unannotated maps; CI scripts deleting and recreating maps with a gap; namespace teardown while apps still run.

Related errors

AI-assisted analysis of go-kratos/kratos@668db92c2c (2026-08-16). Data as JSON: /api/errors/03a83516a32aab69. Report an issue: GitHub.