go-kratos/kratos · info

watcher stopped

Error message

watcher stopped

What it means

ErrWatcherStopped (contrib/registry/zookeeper/watcher.go:16) is returned by watcher.Next when the watcher's internal context is done (the '<-w.ctx.Done()' case around line 68-72). That context is cancelled by watcher.Stop()/Close or by cancellation of the context the watcher was created with, so the error means the watch lifecycle ended and Next must not be called again - it is a termination sentinel, not a ZooKeeper session event.

Source

Thrown at contrib/registry/zookeeper/watcher.go:16

package zookeeper

import (
	"context"
	"errors"
	"path"
	"sync/atomic"

	"github.com/go-zookeeper/zk"

	"github.com/go-kratos/kratos/v3/registry"
)

var _ registry.Watcher = (*watcher)(nil)

var ErrWatcherStopped = errors.New("watcher stopped")

type watcher struct {
	ctx    context.Context
	event  chan zk.Event
	conn   *zk.Conn
	cancel context.CancelFunc

	first atomic.Bool
	// prefix for ZooKeeper paths or keys (used for filtering or identifying watched nodes)
	prefix string
	// the name of the service being watched in ZooKeeper
	serviceName string
}

func newWatcher(ctx context.Context, prefix, serviceName string, conn *zk.Conn) (*watcher, error) {
	w := &watcher{conn: conn, event: make(chan zk.Event, 1), prefix: prefix, serviceName: serviceName}
	w.ctx, w.cancel = context.WithCancel(ctx)
	go w.watch(w.ctx)

View on GitHub (pinned to 668db92c2c)

Solutions

1. Break out of the consume loop on errors.Is(err, zookeeper.ErrWatcherStopped) - it is expected during shutdown
2. Pass a long-lived context (app lifetime) to the Watch call, not a per-request context
3. Ensure exactly one owner calls Stop() and the reader goroutine exits on this sentinel

Example fix

// before
for {
    ins, err := w.Next()
    if err != nil { log.Error("zk watch", err) } // spams after Stop
}

// after
for {
    ins, err := w.Next()
    if errors.Is(err, zookeeper.ErrWatcherStopped) {
        return // watcher closed, exit loop
    }
    if err != nil { return err }
    update(ins)
}

ins, err := w.Next()
if err != nil {
    if errors.Is(err, zookeeper.ErrWatcherStopped) {
        return nil // watcher stopped: exit loop, do not restart
    }
    return err
}

When it happens

Trigger: Calling w.Next() after w.Stop(); the parent context passed at watch creation being cancelled (request context timeout, app shutdown) which cancels w.ctx; kratos' registry watcher loop tearing down during server Close(). Contrast with zk session/Event errors, which arrive on the event channel rather than as this sentinel.

Common situations: App shutdown where the goroutine draining the watcher outlives Stop(); passing a short-lived request ctx into registry.Watch instead of the app's lifetime context; test code that stops the watcher and then performs one more Next assertion.

Related errors

• iterator closed
• watch context canceled: %v
• key not found
• register failed: instance duplicated:
• Discovery.GetService fetch failed

AI-assisted analysis of go-kratos/kratos@668db92c2c (2026-08-16). Data as JSON: /api/errors/98deea8a2612b71f. Report an issue: GitHub.