crowdsecurity/crowdsec · error

consumer %s is not deregistered after %d tries

Error message

consumer %s is not deregistered after %d tries

What it means

Returned by WaitForConsumerDeregistration when the consumer never disappears (or never errors) after `MaxRetries` DescribeStreamConsumer polls with increasing 200ms*i backoff. AWS guarantees deletion is quick but asynchronous; this means it stayed visible past the polling budget.

Source

Thrown at pkg/acquisition/modules/kinesis/run.go:90

		_, err := s.kClient.DescribeStreamConsumer(ctx, &kinesis.DescribeStreamConsumerInput{
				ConsumerName: aws.String(consumerName),
				StreamARN:    aws.String(streamARN),
			})

		var resourceNotFoundErr *kinTypes.ResourceNotFoundException
		if errors.As(err, &resourceNotFoundErr) {
			return nil
		}

		if err != nil {
			s.logger.Errorf("Error while waiting for consumer deregistration: %s", err)
			return fmt.Errorf("cannot describe stream consumer: %w", err)
		}

		time.Sleep(time.Millisecond * 200 * time.Duration(i+1))
	}

	return fmt.Errorf("consumer %s is not deregistered after %d tries", consumerName, maxTries)
}

func (s *Source) DeregisterConsumer(ctx context.Context) error {
	s.logger.Debugf("Deregistering consumer %s if it exists", s.Config.ConsumerName)
	_, err := s.kClient.DeregisterStreamConsumer(ctx, &kinesis.DeregisterStreamConsumerInput{
			ConsumerName: aws.String(s.Config.ConsumerName),
			StreamARN:    aws.String(s.Config.StreamARN),
		})

	var resourceNotFoundErr *kinTypes.ResourceNotFoundException
	if errors.As(err, &resourceNotFoundErr) {
		return nil
	}

	if err != nil {
		return fmt.Errorf("cannot deregister stream consumer: %w", err)
	}

View on GitHub (pinned to 909b515798)

Solutions

1. Increase `max_retries` in the kinesis acquisition config to extend the polling window.
2. Wait a few seconds and re-run; consumer deletion is asynchronous and usually completes shortly after.
3. Verify with `aws kinesis describe-stream-consumer --consumer-name <name> --stream-arn <arn>` whether the consumer is genuinely stuck, and contact AWS support if DELETING persists for minutes.
4. Check the region configured matches the stream's region so polling hits the right endpoint.

Example fix

// before
max_retries: 3
// after
max_retries: 10

// Before deregistering, confirm consumer state
out, err := client.DescribeStreamConsumer(ctx, &kinesis.DescribeStreamConsumerInput{ConsumerARN: aws.String(arn)})
if err == nil && out.ConsumerDescription.ConsumerStatus != nil && *out.ConsumerDescription.ConsumerStatus == "DELETING" {
    // deletion already in flight; allow extra polling budget
}

err := waitForDeregistration(ctx, name, arn)
if err != nil {
    var to *TimeoutError
    if errors.As(err, &to) { logger.Warn("consumer deletion still in flight; proceeding idempotently") }
}

When it happens

Trigger: DeregisterConsumer succeeded but repeated DescribeStreamConsumer calls keep returning the consumer with status DELETING for maxTries iterations (default MaxRetries from config).

Common situations: Heavily loaded streams where deletion is slow; very low `max_retries` in config; region mismatch causing polls against a copy where the consumer still exists (unlikely but possible with misconfigured endpoints).

Understand the failure class

Background: Request timed out: what client-side request timeouts mean across libraries (Request timed out, TIMED_OUT, APITimeoutError) — this error's family across 39 libraries.

Related errors

• cannot wait for consumer deregistration: %w
• consumer %s is not active after %d tries
• timeout while waiting for consumer to be active: %w
• stream_name is mandatory when use_enhanced_fanout is false
• stream_arn is mandatory when use_enhanced_fanout is true

AI-assisted analysis of crowdsecurity/crowdsec@909b515798 (2026-09-06). Data as JSON: /api/errors/bf05d8dec75b2faa. Report an issue: GitHub.