kubernetes/kops · error

failed to generate AWS IAM S3 access statements: %v

Error message

failed to generate AWS IAM S3 access statements: %v

What it means

Returned by NodeRoleAPIServer.BuildAWSPolicy when PolicyBuilder.AddS3Permissions fails while generating the S3 access statements for the kube-apiserver instance role. AddS3Permissions converts the cluster's state store paths into IAM S3 statements; any VFS parse or unsupported-backend failure there is wrapped with this message. The policy generation for the API server role is then aborted.

Source

Thrown at pkg/model/iam/iam_builder.go:393

		clusterName:               clusterName,
		region:                    region,
		unconditionalAction:       sets.New[string](),
		clusterTaggedAction:       sets.New[string](),
		clusterTaggedCreateAction: sets.New[string](),
		kmsDataPlaneAction:        sets.New[string](),
		partition:                 partition,
	}
	return p
}

// BuildAWSPolicy generates a custom policy for a Kubernetes master.
func (r *NodeRoleAPIServer) BuildAWSPolicy(b *PolicyBuilder) (*Policy, error) {
	p := NewPolicy(b.Cluster.GetName(), b.Partition, b.Region)

	b.addNodeupPermissions(p, r.warmPool)

	if err := b.AddS3Permissions(p); err != nil {
		return nil, fmt.Errorf("failed to generate AWS IAM S3 access statements: %v", err)
	}

	// The API server role may host a kms-plugin sidecar wired to the instance role
	// when EncryptionConfig is enabled; bypass kms:ViaService so that direct KMS
	// calls from kube-apiserver are not denied.
	addKMSIAMPolicies(p, fi.ValueOf(b.Cluster.Spec.EncryptionConfig))

	if b.Cluster.Spec.IAM != nil && b.Cluster.Spec.IAM.AllowContainerRegistry {
		addECRPermissions(p)
	}

	if b.Cluster.Spec.Containerd != nil && b.Cluster.Spec.Containerd.UseECRCredentialsForMirrors {
		addECRPullThroughPermissions(p)
	}

	if b.Cluster.Spec.Networking.AmazonVPC != nil {
		addAmazonVPCCNIPermissions(p)
	}

View on GitHub (pinned to 4c8573c808)

Solutions

1. Read the inner error for the exact failing path and cause.
2. Confirm the state store is a valid s3:// URL: `kops get cluster -o yaml` and inspect spec.configStore.base.
3. Correct the configStore.base (and etcd backupStore if malformed) in the cluster spec, then rerun `kops update cluster`.
4. If intentionally using another cloud's storage, do not build AWS IAM policies for it — use the matching provider's model.

Example fix

// before
spec:
  configStore:
    base: gs://my-kops-bucket/cluster.example.com
// after
spec:
  configStore:
    base: s3://my-kops-bucket/cluster.example.com

// Validate configStore.base is parseable and S3 before building policies
if _, err := vfs.Context.BuildVfsPath(cluster.Spec.ConfigStore.Base); err != nil {
    return fmt.Errorf("bad configStore.base %q: %w", cluster.Spec.ConfigStore.Base, err)
}

policy, err := apiServerRole.BuildAWSPolicy(builder)
if err != nil {
    return nil, fmt.Errorf("api-server IAM policy: %w", err)
}

When it happens

Trigger: Building the NodeRoleAPIServer policy (via PolicyBuilder.BuildAWSPolicy during cluster update/creation) when AddS3Permissions encounters a state store root that vfs.Context.BuildVfsPath cannot parse, or a non-S3/MemFS/FSPath backend in the default switch case.

Common situations: State store pointing at a backend unsupported for AWS IAM synthesis (e.g. gs://, do:// spaces URLs fed into AWS policy build); typo in the state store bucket URL; etcd backupStore with an invalid scheme; running against a cluster whose config was migrated from another cloud provider.

Related errors

• unknown writeable path, can't apply IAM policy: %q
• failed to generate AWS IAM Policy: %v
• cannot parse VFS path %q: %v
• path is not cluster readable: %v
• error building IAM policy: %v

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/8964e7289f2cd76c. Report an issue: GitHub.