crowdsecurity/crowdsec · error

while describing group %s: %w

Error message

while describing group %s: %w

What it means

WatchLogGroupForStreams calls DescribeLogGroups pages to discover streams in the log group. If any page fetch fails (network error, throttling, auth failure, missing permissions) the error is wrapped as `while describing group <name>: <err>`, aborting the streaming acquisition loop.

Source

Thrown at pkg/acquisition/modules/cloudwatch/run.go:134

		case <-s.t.Dying():
			s.logger.Infof("stopping group watch")
			return nil
		case <-ticker.C:
			p := cloudwatchlogs.NewDescribeLogStreamsPaginator(
				s.cwClient,
				&cloudwatchlogs.DescribeLogStreamsInput{
					LogGroupName: aws.String(s.Config.GroupName),
					Descending:   aws.Bool(true),
					OrderBy:      cwTypes.OrderByLastEventTime,
					Limit:        s.Config.DescribeLogStreamsLimit,
				},
				)

			Pageloop:
			for p.HasMorePages() {
				page, err := p.NextPage(ctx)
				if err != nil {
					return fmt.Errorf("while describing group %s: %w", s.Config.GroupName, err)
				}

				for _, event := range page.LogStreams {
					// we check if the stream has been written to recently enough to be monitored
					if event.LastIngestionTime == nil {
						continue
					}

					// aws uses millisecond since the epoch
					oldest := time.Now().UTC().Add(-*s.Config.MaxStreamAge)
					// TBD : verify that this is correct : Unix 2nd arg expects Nanoseconds, and have a code that is more explicit.
					LastIngestionTime := time.Unix(0, *event.LastIngestionTime*int64(time.Millisecond))
					if LastIngestionTime.Before(oldest) {
						s.logger.Tracef("stop iteration, %s reached oldest age, stop (%s < %s)", aws.ToString(event.LogStreamName), LastIngestionTime, time.Now().UTC().Add(-*s.Config.MaxStreamAge))
						break Pageloop
					}

					var expectMode int

View on GitHub (pinned to 909b515798)

Solutions

1. Check IAM permissions include logs:DescribeLogGroups for the group ARN
2. Retry on transient network errors; consider restarting crowdsec — the loop stops on first error
3. Check for throttling in the wrapped error and reduce poll frequency or request a quota increase
4. Verify aws_region matches where the log group lives

Example fix

// before
{"log_group": "/aws/lambda/missing-perms"}
// after — attach policy
{"Effect": "Allow", "Action": "logs:DescribeLogGroups", "Resource": "arn:aws:logs:*:*:log-group:/aws/*"}

_, err := client.DescribeLogGroups(ctx, &cwlogs.DescribeLogGroupsInput{LogGroupNamePrefix: aws.String(group)})
if err != nil { return fmt.Errorf("preflight describe failed: %w", err) }

page, err := p.NextPage(ctx)
if err != nil {
    var tae smithy.APIError
    if errors.As(err, &tae) && tae.ErrorCode() == "ThrottlingException" {
        time.Sleep(backoff); continue // retry with backoff
    }
    return fmt.Errorf("while describing group %s: %w", group, err)
}

When it happens

Trigger: StreamingAcquisition calls WatchLogGroupForStreams and a p.NextPage(ctx) call fails — network interruption, AWS throttling (rate exceeded), IAM lacking logs:DescribeLogGroups, or the group region/profile being wrong.

Common situations: Long-running streaming hitting transient AWS API errors or rate limits; IAM policy missing logs:DescribeLogGroups on the group; wrong region configured so the group lookup fails mid-poll.

Understand the failure class

Background: "API request failed": what wrapped HTTP errors from external APIs mean and how to find the real cause — this error's family across 29 libraries.

Related errors

• while reading %s/%s: %w
• while reading logs from %s/%s: %w
• cannot list shards for enhanced_read: %w
• group_name is mandatory for CloudwatchSource
• aws_region is not specified, specify it or aws_config_dir

AI-assisted analysis of crowdsecurity/crowdsec@909b515798 (2026-09-06). Data as JSON: /api/errors/5fc50665c3951e65. Report an issue: GitHub.