crowdsecurity/crowdsec · error

cannot parse stream ARN: %w

Error message

cannot parse stream ARN: %w

What it means

The stream_arn configured for the kinesis datasource could not be parsed by the aws-sdk arn.Parse function. This is a local validation error thrown before any AWS call is made, meaning the configured value is not a syntactically valid Amazon Resource Name (missing arn: prefix, wrong number of ':'-separated fields, empty parts).

Source

Thrown at pkg/acquisition/modules/kinesis/run.go:276

				StartingPosition: &kinTypes.StartingPosition{Type: kinTypes.ShardIteratorTypeLatest},
				ConsumerARN:      streamConsumer.Consumer.ConsumerARN,
			})
		if err != nil {
			return fmt.Errorf("cannot subscribe to shard: %w", err)
		}

		s.shardReaderTomb.Go(func() error {
			return s.ReadFromSubscription(r.GetStream().Reader, out, shardID, arn.Resource[7:])
		})
	}

	return nil
}

func (s *Source) EnhancedRead(ctx context.Context, out chan pipeline.Event, t *tomb.Tomb) error {
	parsedARN, err := arn.Parse(s.Config.StreamARN)
	if err != nil {
		return fmt.Errorf("cannot parse stream ARN: %w", err)
	}

	if !strings.HasPrefix(parsedARN.Resource, "stream/") {
		return fmt.Errorf("resource part of stream ARN %s does not start with stream/", s.Config.StreamARN)
	}

	s.logger = s.logger.WithField("stream", parsedARN.Resource[7:])
	s.logger.Info("starting kinesis acquisition with enhanced fan-out")

	err = s.DeregisterConsumer(ctx)
	if err != nil {
		return fmt.Errorf("cannot deregister consumer: %w", err)
	}

	streamConsumer, err := s.RegisterConsumer(ctx)
	if err != nil {
		return fmt.Errorf("cannot register consumer: %w", err)
	}

View on GitHub (pinned to 909b515798)

Solutions

1. Set a complete, valid ARN: arn:aws:kinesis:<region>:<account-id>:stream/<stream-name>.
2. Quote the ARN in YAML if it contains special characters.
3. Check for copy/paste artifacts (newlines, BOM, spaces) in the config value.
4. If enhanced fan-out is not desired, remove the incomplete stream_arn setting and use stream_name + classic polling instead.

Example fix

// before (acquis.yaml)
stream_arn: my-stream
// after
stream_arn: "arn:aws:kinesis:eu-west-1:123456789012:stream/my-stream"

func validStreamARN(s string) bool {
	return strings.HasPrefix(s, "arn:aws:kinesis:") && strings.Contains(s, ":stream/")
}
if !validStreamARN(cfg.StreamARN) {
	return fmt.Errorf("stream_arn %q is not a kinesis stream ARN", cfg.StreamARN)
}

When it happens

Trigger: EnhancedRead is entered (enhanced fan-out enabled) and s.Config.StreamARN is empty, contains spaces/quotes, uses the wrong number of colons (arn:aws:kinesis:region:acct:stream requires exactly 5 colons after 'arn:'), or a non-ARN string like just the stream name was supplied.

Common situations: User put the stream name instead of the full ARN in acquis.yaml; YAML indentation mangled the value; ARN pasted with extra characters or line breaks; user omitted stream_arn entirely while enabling enhanced fan-out.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• resource part of stream ARN %s does not start with stream/
• group_name is mandatory for CloudwatchSource
• path must start with /
• basic_auth is selected, but basic_auth is not provided
• basic_auth is selected, but username is not provided

AI-assisted analysis of crowdsecurity/crowdsec@909b515798 (2026-09-06). Data as JSON: /api/errors/edef476a0aaa16db. Report an issue: GitHub.