crowdsecurity/crowdsec · error

failed to load aws config: %w

Error message

failed to load aws config: %w

What it means

Wraps any failure from the AWS SDK v2 `config.LoadDefaultConfig` when the kinesis source builds its client in `newClient`. The SDK resolves region, credentials and shared config (profile files, env vars, IMDS) at this point; a failure here means the AWS configuration chain itself could not be assembled, before any Kinesis API call is made. The source aborts configuration and CrowdSec will not start this acquisition.

Source

Thrown at pkg/acquisition/modules/kinesis/config.go:133

	var loadOpts []func(*config.LoadOptions) error
	if s.Config.AwsProfile != nil && *s.Config.AwsProfile != "" {
		loadOpts = append(loadOpts, config.WithSharedConfigProfile(*s.Config.AwsProfile))
	}

	region := s.Config.AwsRegion
	if region == "" {
		region = "us-east-1"
	}

	loadOpts = append(loadOpts, config.WithRegion(region))

	if c := defaultCreds(); c != nil {
		loadOpts = append(loadOpts, config.WithCredentialsProvider(c))
	}

	cfg, err := config.LoadDefaultConfig(ctx, loadOpts...)
	if err != nil {
		return fmt.Errorf("failed to load aws config: %w", err)
	}

	var clientOpts []func(*kinesis.Options)
	if s.Config.AwsEndpoint != "" {
		clientOpts = append(clientOpts, func(o *kinesis.Options) {
			o.BaseEndpoint = aws.String(s.Config.AwsEndpoint)
		})
	}

	s.kClient = kinesis.NewFromConfig(cfg, clientOpts...)

	return nil
}

View on GitHub (pinned to 909b515798)

Solutions

1. Verify the `aws_profile` value in the kinesis acquisition YAML matches a section in ~/.aws/config and ~/.aws/credentials (run `aws configure list --profile <name>`).
2. Check AWS_CONFIG_FILE / AWS_SHARED_CREDENTIALS_FILE env vars point to existing, readable files, and that the files parse as valid INI.
3. If no profile is set, run `aws configure` or set AWS_REGION and credentials env vars so LoadDefaultConfig has a valid default chain.
4. If overriding credentials via defaultCreds, confirm the returned provider can be constructed without error.

Example fix

// before
profile: "prod-rea"
// after (profile exists in ~/.aws/config)
profile: "prod-reader"

// Go, before starting acquisition
if _, err := os.Stat(filepath.Join(mustHome(), ".aws", "credentials")); err != nil {
    return fmt.Errorf("AWS shared credentials file missing: %w", err)
}
if profile != "" {
    cfg, err := ini.Load(sharedConfigPath())
    if err != nil || !cfg.HasSection("profile " + profile) && !cfg.HasSection(profile) {
        return fmt.Errorf("aws profile %q not found in shared config", profile)
    }
}

cfg, err := config.LoadDefaultConfig(ctx, loadOpts...)
if err != nil {
    var cfgErr *aws.ConfigError
    if errors.As(err, &cfgErr) {
        logger.Fatalf("bad AWS config: %v", cfgErr)
    }
    return fmt.Errorf("failed to load aws config: %w", err)
}

When it happens

Trigger: Calling Source.Configure with an `aws_profile` that has no matching section in ~/.aws/config or ~/.aws/credentials; a malformed shared credentials/config file; or a credentials provider (e.g. a custom one injected via defaultCreds) that fails to load. Note the SDK normally defers credential fetch to first API call, so this mostly fires on config-file/profile parse errors.

Common situations: Typo in `aws_profile` name in the acquisition YAML; corrupted or wrong-permission ~/.aws/credentials; missing shared config file referenced by AWS_CONFIG_FILE; running in a container where the mounted AWS config is absent.

Understand the failure class

Background: "environment variable is not set" and "Missing keys in environment" errors: what missing required env var messages mean and how to fix them — this error's family across 28 libraries.

Related errors

• group_name is mandatory for CloudwatchSource
• aws_region is not specified, specify it or aws_config_dir
• stream_name is mandatory when use_enhanced_fanout is false
• stream_arn is mandatory when use_enhanced_fanout is true
• consumer_name is mandatory when use_enhanced_fanout is true

AI-assisted analysis of crowdsecurity/crowdsec@909b515798 (2026-09-06). Data as JSON: /api/errors/48f5483de5a5522c. Report an issue: GitHub.