{"record":{"id":"ccc74519f0c65de4","repo":"crowdsecurity/crowdsec","slug":"cannot-list-shards-for-enhanced-read-w","errorCode":null,"errorMessage":"cannot list shards for enhanced_read: %w","messagePattern":"cannot list shards for enhanced_read: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/acquisition/modules/kinesis/run.go","lineNumber":250,"sourceCode":"\t\t\t\treturn nil\n\t\t\t}\n\n\t\t\tswitch et := event.(type) {\n\t\t\tcase *kinTypes.SubscribeToShardEventStreamMemberSubscribeToShardEvent:\n\t\t\t\ts.ParseAndPushRecords(et.Value.Records, out, logger, shardID)\n\t\t\tdefault:\n\t\t\t\tlogger.Infof(\"unhandled SubscribeToShard event: %T\", et)\n\t\t\t}\n\t\t}\n\t}\n}\n\nfunc (s *Source) SubscribeToShards(ctx context.Context, arn arn.ARN, streamConsumer *kinesis.RegisterStreamConsumerOutput, out chan pipeline.Event) error {\n\tshards, err := s.kClient.ListShards(ctx, &kinesis.ListShardsInput{\n\t\t\tStreamName: aws.String(arn.Resource[7:]),\n\t\t})\n\tif err != nil {\n\t\treturn fmt.Errorf(\"cannot list shards for enhanced_read: %w\", err)\n\t}\n\n\tfor _, shard := range shards.Shards {\n\t\tshardID := *shard.ShardId\n\n\t\tr, err := s.kClient.SubscribeToShard(ctx, &kinesis.SubscribeToShardInput{\n\t\t\t\tShardId:          aws.String(shardID),\n\t\t\t\tStartingPosition: &kinTypes.StartingPosition{Type: kinTypes.ShardIteratorTypeLatest},\n\t\t\t\tConsumerARN:      streamConsumer.Consumer.ConsumerARN,\n\t\t\t})\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"cannot subscribe to shard: %w\", err)\n\t\t}\n\n\t\ts.shardReaderTomb.Go(func() error {\n\t\t\treturn s.ReadFromSubscription(r.GetStream().Reader, out, shardID, arn.Resource[7:])\n\t\t})\n\t}","sourceCodeStart":232,"sourceCodeEnd":268,"githubUrl":"https://github.com/crowdsecurity/crowdsec/blob/909b5157986a2b2c2163300fdaef5ed01289f7d2/pkg/acquisition/modules/kinesis/run.go#L232-L268","documentation":"This error wraps any failure returned by the AWS Kinesis ListShards API while the source is enumerating the shards of a stream in enhanced fan-out mode. ListShards fails when the request cannot be served — typically because the stream does not exist, the IAM principal lacks kinesis:ListShards/DescribeStreamSummary permission, or the AWS credentials/region are wrong. It is thrown by SubscribeToShards, the entry point for enhanced fan-out shard enumeration, and bubbles up as 'cannot subscribe to shards' through EnhancedRead.","triggerScenarios":"Calling SubscribeToShards (via EnhancedRead) when: the StreamARN names a stream that was deleted or is in a different region/account; the IAM role lacks kinesis:ListShards or kinesis:DescribeStreamSummary; credentials are expired, missing, or point to the wrong profile; the stream is in CREATING/DELETING status; or network/DNS blocks the Kinesis endpoint.","commonSituations":"Typos in stream_arn in the acquis.yaml kinesis source; running crowdsec on an instance whose IAM role was changed; region mismatch between the configured ARN and AWS_REGION; deleting/rescaling the stream while the datasource starts; corporate egress firewall dropping kinesis.<region>.amazonaws.com.","solutions":["Verify the stream exists and is ACTIVE in the configured region: aws kinesis describe-stream-summary --stream-name <name>","Fix the stream_arn in the crowdsec acquisition config (wrong region/account is the most common cause).","Check IAM permissions: the principal needs kinesis:ListShards (and DescribeStreamSummary) on the stream resource.","Validate credentials (aws sts get-caller-identity) and that AWS_REGION/profile match the ARN.","Retry on transient failures; if it persists, check network reachability to the Kinesis endpoint."],"exampleFix":"// before: ambiguous ARN region\nstream_arn: arn:aws:kinesis:us-west-2:111122223333:stream/my-stream\n// after: region corrected to match the actual stream\nstream_arn: arn:aws:kinesis:us-east-1:111122223333:stream/my-stream","handlingStrategy":"validation","validationCode":"// Pre-flight before starting the datasource\nout, err := awsClient.DescribeStreamSummary(ctx, &kinesis.DescribeStreamSummaryInput{\n\tStreamName: aws.String(streamNameFromARN),\n})\nif err != nil || *out.StreamDescriptionSummary.StreamStatus != \"ACTIVE\" {\n\treturn fmt.Errorf(\"stream %s not usable: %w\", streamName, err)\n}","typeGuard":null,"tryCatchPattern":"var opErr *types.Error\nif err := runKinesis(); err != nil {\n\tif errors.As(err, &opErr) {\n\t\tlog.Printf(\"kinesis API error: %s\", opErr.ErrorCode()) // ResourceNotFound, AccessDenied...\n\t}\n\treturn err\n}","preventionTips":["Validate the stream ARN and stream status before launching crowdsec with the kinesis datasource.","Attach a least-privilege IAM policy covering ListShards, DescribeStreamSummary, SubscribeToShard, Register/DeregisterStreamConsumer.","Pin AWS_REGION explicitly in the service unit/environment.","Smoke-test credentials with aws sts get-caller-identity on the host."],"tags":["aws","kinesis","network","iam"],"backgroundTag":"api-request-failed","analyzedSha":"909b5157986a2b2c2163300fdaef5ed01289f7d2","analyzedAt":"2026-09-06T12:27:26.012Z","contentChangedAt":"2026-09-06T12:27:26.012Z","schemaVersion":2},"datasetVersion":"2026-09-14T05:17:10.506Z"}