client stopped and may not longer create config entries

Error message

client stopped and may not longer create config entries

What it means

Nomad's Consul config-entry wrapper (setCE) guards writes with a stopped flag. Once the background deletion goroutine has been shut down, any attempt to create a Consul config entry (ingress/terminating-gateway) returns this error instead of racing a dead worker. The typo 'may not longer' is in the original message.

Source

Thrown at nomad/consul.go:102

func (c *consulConfigsAPI) SetIngressCE(ctx context.Context, namespace, service, cluster, partition string, entry *structs.ConsulIngressConfigEntry) error {
	return c.setCE(ctx, convertIngressCE(namespace, service, entry), cluster, partition)
}

func (c *consulConfigsAPI) SetTerminatingCE(ctx context.Context, namespace, service, cluster, partition string, entry *structs.ConsulTerminatingConfigEntry) error {
	return c.setCE(ctx, convertTerminatingCE(namespace, service, entry), cluster, partition)
}

// setCE will set the Configuration Entry of any type Consul supports.
func (c *consulConfigsAPI) setCE(ctx context.Context, entry api.ConfigEntry, cluster, partition string) error {
	defer metrics.MeasureSince([]string{"nomad", "consul", "create_config_entry"}, time.Now())

	// make sure the background deletion goroutine has not been stopped
	c.lock.Lock()
	stopped := c.stopped
	c.lock.Unlock()

	if stopped {
		return errors.New("client stopped and may not longer create config entries")
	}

	// ensure we are under our wait limit
	if err := c.limiter.Wait(ctx); err != nil {
		return err
	}

	client := c.configsClientFunc(cluster)
	_, _, err := client.Set(entry, &api.WriteOptions{
		Namespace: entry.GetNamespace(),
		Partition: partition,
	})
	return err
}

func convertIngressCE(namespace, service string, entry *structs.ConsulIngressConfigEntry) api.ConfigEntry {
	var listeners []api.IngressListener = nil
	for _, listener := range entry.Listeners {

View on GitHub (pinned to 482b49bf1a)

Solutions

Retry the operation after Nomad restarts and the consul client is re-initialized
Ensure config-entry writes are issued while the server is healthy, not during shutdown
Check server logs for the shutdown sequence to confirm this was a race with process exit
If seen persistently while the server is up, investigate why the deletion goroutine stopped early (bug report)

Example fix

// before
err := consulClient.SetIngressCE(ctx, service, namespace, gateway, config)
// after: check server health before writing
if err := apiClient.Agent().HealthChecks...; nomadRunning {
    err = consulClient.SetIngressCE(ctx, service, namespace, gateway, config)
}

Defensive patterns

Strategy: retry

Validate before calling

// before writing config entries, ensure nomad is not shutting down
if serverShuttingDown() { return fmt.Errorf("defer config entry write") }

Try / catch

// Go
if err := c.SetIngressCE(ctx, svc, ns, gw, cfg); err != nil {
    if strings.Contains(err.Error(), "client stopped") {
        // shutdown race: requeue after restart
        requeueAfterRestart(cfg)
        return nil
    }
    return err
}

Prevention

Avoid issuing consul config-entry writes during deployments/shutdowns
Requeue writes on restart via a reconciliation loop
Check server health endpoints before automation runs
Watch server logs for the deletion-goroutine stop message

When it happens

Trigger: Calling SetIngressCE or SetTerminatingCE after the client's consul config-entry deletion goroutine was stopped during Nomad shutdown; a config-entry write racing server shutdown.

Common situations: Ingress/terminating-gateway config entries being applied while the Nomad server is shutting down or being restarted; tests or scripts pushing config entries at shutdown time.

Related errors

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/b275823bf114a1c9. Report an issue: GitHub.