crowdsecurity/crowdsec · error

timeout exceeded

Error message

timeout exceeded

What it means

GRPCClient.Notify runs the plugin call in a goroutine and selects on the context. When ctx expires before the plugin responds (plugins run as separate subprocesses over gRPC and can be slow), Notify returns 'timeout exceeded'. It signals the caller that the notification was not delivered in time.

Source

Thrown at pkg/csplugin/notifier.go:36

type GRPCClient struct{
	protobufs.UnimplementedNotifierServer
	client protobufs.NotifierClient 
}

func (m *GRPCClient) Notify(ctx context.Context, notification *protobufs.Notification) (*protobufs.Empty, error) {
	done := make(chan error)
	go func() {
		_, err := m.client.Notify(
			ctx, &protobufs.Notification{Text: notification.GetText(), Name: notification.GetName()},
		)
		done <- err
	}()
	select {
	case err := <-done:
		return &protobufs.Empty{}, err

	case <-ctx.Done():
		return &protobufs.Empty{}, errors.New("timeout exceeded")
	}
}

func (m *GRPCClient) Configure(ctx context.Context, config *protobufs.Config) (*protobufs.Empty, error) {
	_, err := m.client.Configure(ctx, config)
	return &protobufs.Empty{}, err
}

type GRPCServer struct {
	Impl protobufs.NotifierServer
}

func (p *NotifierPlugin) GRPCServer(broker *plugin.GRPCBroker, s *grpc.Server) error {
	protobufs.RegisterNotifierServer(s, p.Impl)
	return nil
}

func (*NotifierPlugin) GRPCClient(ctx context.Context, broker *plugin.GRPCBroker, c *grpc.ClientConn) (any, error) {

View on GitHub (pinned to 909b515798)

Solutions

1. Check the plugin process logs for hangs and test the plugin's external target (webhook URL, SMTP server) for reachability
2. Increase the plugin's 'timeout' setting in its config (defaults to 5s in the broker)
3. Restart crowdsec to respawn the plugin subprocess if it is stuck

Example fix

// before (plugin config.yaml)
slack:
  type: slack
  webhook_url: https://hooks.slack.com/...

// after
slack:
  type: slack
  webhook_url: https://hooks.slack.com/...
  timeout: 30s

ctx, cancel := context.WithTimeout(context.Background(), 30*time.Second)
defer cancel() // allow slow plugins a generous budget

if _, err := client.Notify(ctx, req); err != nil {
    if ctx.Err() == context.DeadlineExceeded || strings.Contains(err.Error(), "timeout exceeded") {
        log.Warn("plugin notification timed out; will retry")
        return retryNotify(req)
    }
    return err
}

When it happens

Trigger: A notification plugin binary that hangs, blocks on a slow network call (e.g. webhook to an unreachable host), or fails to respond within the context deadline passed to Notify.

Common situations: Plugin calling an external API behind a dead firewall; plugin binary crashed/hung; container host under load so the plugin subprocess starts slowly; default plugin timeout (5s from broker config) too short for a slow HTTP sink.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

• Timeouts: ETIMEDOUT, deadlines, and hung requests — what actually expires when a request times out.

Related errors

• loading plugin: %w
• while configuring %s: %w
• plugin %q: notifier not registered
• plugins are enabled, but the plugin_config section is missin
• plugins are enabled, but config_paths.plugin_dir is not defi

AI-assisted analysis of crowdsecurity/crowdsec@909b515798 (2026-09-06). Data as JSON: /api/errors/9ca456aa5d87ba3c. Report an issue: GitHub.