crowdsecurity/crowdsec · error

cannot wait for consumer deregistration: %w

Error message

cannot wait for consumer deregistration: %w

What it means

Returned by DeregisterConsumer when the follow-up WaitForConsumerDeregistration poll (error 421/422) fails, after the DeregisterStreamConsumer call itself succeeded. It signals the deregistration could not be confirmed before EnhancedRead continues.

Source

Thrown at pkg/acquisition/modules/kinesis/run.go:111

func (s *Source) DeregisterConsumer(ctx context.Context) error {
	s.logger.Debugf("Deregistering consumer %s if it exists", s.Config.ConsumerName)
	_, err := s.kClient.DeregisterStreamConsumer(ctx, &kinesis.DeregisterStreamConsumerInput{
			ConsumerName: aws.String(s.Config.ConsumerName),
			StreamARN:    aws.String(s.Config.StreamARN),
		})

	var resourceNotFoundErr *kinTypes.ResourceNotFoundException
	if errors.As(err, &resourceNotFoundErr) {
		return nil
	}

	if err != nil {
		return fmt.Errorf("cannot deregister stream consumer: %w", err)
	}

	err = s.WaitForConsumerDeregistration(ctx, s.Config.ConsumerName, s.Config.StreamARN)
	if err != nil {
		return fmt.Errorf("cannot wait for consumer deregistration: %w", err)
	}

	return nil
}

func (s *Source) WaitForConsumerRegistration(ctx context.Context, consumerARN string) error {
	maxTries := s.Config.MaxRetries
	for i := range maxTries {
		describeOutput, err := s.kClient.DescribeStreamConsumer(ctx, &kinesis.DescribeStreamConsumerInput{
				ConsumerARN: aws.String(consumerARN),
			})
		if err != nil {
			return fmt.Errorf("cannot describe stream consumer: %w", err)
		}

		if describeOutput.ConsumerDescription.ConsumerStatus == "ACTIVE" {
			s.logger.Debugf("Consumer %s is active", consumerARN)
			return nil

View on GitHub (pinned to 909b515798)

Solutions

1. Increase `max_retries` to give deletion confirmation more time.
2. Ensure the IAM principal has `kinesis:DescribeStreamConsumer` so the confirmation polls can run.
3. Check the wrapped error to distinguish a polling failure (permissions/throttle) from a timeout, and act accordingly (see errors for run.go:84 and run.go:90).
4. Re-run acquisition later — deregistration is idempotent, so restarting is safe.

Example fix

// before
max_retries: 3
// after
max_retries: 15

// Pre-check permissions for the confirmation loop
aws iam simulate-principal-policy --policy-source-arn <role> --action-names kinesis:DescribeStreamConsumer kinesis:DeregisterStreamConsumer

if err := src.DeregisterConsumer(ctx); err != nil {
    logger.WithError(err).Warn("deregistration unconfirmed; safe to retry on next start")
    // fall through — deregistration is idempotent (ResourceNotFound short-circuit)
}

When it happens

Trigger: DeregisterStreamConsumer returns 200, then the confirmation loop either errors (throttling, IAM, network — see 421) or times out still seeing the consumer (see 422).

Common situations: Slow consumer deletion on busy streams combined with a low max_retries; describe-permission missing so every poll fails.

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Related errors

• consumer %s is not deregistered after %d tries
• consumer %s is not active after %d tries
• timeout while waiting for consumer to be active: %w
• stream_name is mandatory when use_enhanced_fanout is false
• stream_arn is mandatory when use_enhanced_fanout is true

AI-assisted analysis of crowdsecurity/crowdsec@909b515798 (2026-09-06). Data as JSON: /api/errors/371dd8e8b53f49f0. Report an issue: GitHub.