crowdsecurity/crowdsec · error

resource part of stream ARN %s does not start with stream/

Error message

resource part of stream ARN %s does not start with stream/

What it means

The stream_arn parsed as a valid ARN, but its resource segment is not a Kinesis stream. EnhancedRead requires an ARN of the form arn:aws:kinesis:<region>:<account>:stream/<name> because it later slices Resource[7:] to recover the stream name; any other resource type (e.g. a consumer ARN, or stream/consumer/<name>) fails this prefix check.

Source

Thrown at pkg/acquisition/modules/kinesis/run.go:280

			return fmt.Errorf("cannot subscribe to shard: %w", err)
		}

		s.shardReaderTomb.Go(func() error {
			return s.ReadFromSubscription(r.GetStream().Reader, out, shardID, arn.Resource[7:])
		})
	}

	return nil
}

func (s *Source) EnhancedRead(ctx context.Context, out chan pipeline.Event, t *tomb.Tomb) error {
	parsedARN, err := arn.Parse(s.Config.StreamARN)
	if err != nil {
		return fmt.Errorf("cannot parse stream ARN: %w", err)
	}

	if !strings.HasPrefix(parsedARN.Resource, "stream/") {
		return fmt.Errorf("resource part of stream ARN %s does not start with stream/", s.Config.StreamARN)
	}

	s.logger = s.logger.WithField("stream", parsedARN.Resource[7:])
	s.logger.Info("starting kinesis acquisition with enhanced fan-out")

	err = s.DeregisterConsumer(ctx)
	if err != nil {
		return fmt.Errorf("cannot deregister consumer: %w", err)
	}

	streamConsumer, err := s.RegisterConsumer(ctx)
	if err != nil {
		return fmt.Errorf("cannot register consumer: %w", err)
	}

	for {
		s.shardReaderTomb = &tomb.Tomb{}

View on GitHub (pinned to 909b515798)

Solutions

1. Use the Kinesis Data Stream ARN, not a consumer or Firehose ARN: arn:aws:kinesis:<region>:<acct>:stream/<name>.
2. Verify with aws kinesis describe-stream-summary --stream-arn <arn> that the ARN is accepted as a stream.
3. Check that nothing stripped 'stream/' from the configured value.

Example fix

// before: consumer ARN mistakenly used
stream_arn: arn:aws:kinesis:eu-west-1:123456789012:stream/my-stream/consumer/crowdsec:1234
// after: stream ARN
stream_arn: arn:aws:kinesis:eu-west-1:123456789012:stream/my-stream

parsed, err := arn.Parse(cfg.StreamARN)
if err == nil && !strings.HasPrefix(parsed.Resource, "stream/") {
	return fmt.Errorf("expected stream ARN, got resource %q", parsed.Resource)
}

When it happens

Trigger: EnhancedRead with StreamARN set to a kinesis consumer ARN (resource 'stream-consumer/...'), a Firehose delivery stream ARN, or a truncated ARN where the resource doesn't start with 'stream/'.

Common situations: User copied the enhanced-consumer ARN (from aws kinesis list-stream-consumers) instead of the stream ARN; user pointed the datasource at a Firehose or Data Streams resource of another type; hand-edited ARN dropped the 'stream/' part.

Understand the failure class

Background: "Invalid value" and "allowed values are" config errors: what your library rejected and how to fix it — this error's family across 41 libraries.

Related errors

• cannot parse stream ARN: %w
• group_name is mandatory for CloudwatchSource
• path must start with /
• basic_auth is selected, but basic_auth is not provided
• basic_auth is selected, but username is not provided

AI-assisted analysis of crowdsecurity/crowdsec@909b515798 (2026-09-06). Data as JSON: /api/errors/152e22ca7acedee8. Report an issue: GitHub.