kubernetes/kops · error

error listing Nat Gateways %v

Error message

error listing Nat Gateways %v

What it means

natgateway.go Find() fetches a NAT gateway by its explicit ID via ec2:DescribeNatGateways and wraps any API error with this message. It is thrown only when e.ID is set and the DescribeNatGateways call fails, so the task's actual state cannot be determined.

Source

Thrown at upup/pkg/fi/cloudup/awstasks/natgateway.go:79

}

func (e *NatGateway) Find(c *fi.CloudupContext) (*NatGateway, error) {
	ctx := c.Context()
	cloud := awsup.GetCloud(c)
	var ngw *ec2types.NatGateway
	actual := &NatGateway{}

	if fi.ValueOf(e.ID) != "" {
		// We have an existing NGW, lets look up the EIP
		ngwIds := []string{fi.ValueOf(e.ID)}

		request := &ec2.DescribeNatGatewaysInput{
			NatGatewayIds: ngwIds,
		}

		response, err := cloud.EC2().DescribeNatGateways(ctx, request)
		if err != nil {
			return nil, fmt.Errorf("error listing Nat Gateways %v", err)
		}

		if len(response.NatGateways) != 1 {
			return nil, fmt.Errorf("found %d Nat Gateways with ID %q, expected 1", len(response.NatGateways), fi.ValueOf(e.ID))
		}
		ngw = &response.NatGateways[0]

		if len(ngw.NatGatewayAddresses) != 1 {
			return nil, fmt.Errorf("found %d EIP Addresses for 1 NATGateway, expected 1", len(ngw.NatGatewayAddresses))
		}
	} else {
		// This is the normal/default path
		var err error
		ngw, err = e.findNatGateway(c)
		if err != nil {
			return nil, err
		}
		if ngw == nil {

View on GitHub (pinned to 4c8573c808)

Solutions

  1. Check the wrapped error code to distinguish auth vs throttling vs invalid-ID
  2. Verify the NAT gateway ID in the kOps state exists in the configured region/account (aws ec2 describe-nat-gateways --nat-gateway-ids ngw-...)
  3. Grant ec2:DescribeNatGateways to the kOps IAM role
  4. Retry on throttling
  5. If the NAT gateway was deleted out-of-band, clear the ID from state (kops edit cluster / replace the task) so it can be recreated
Defensive patterns

Strategy: retry

Validate before calling

_, err := cloud.EC2().DescribeNatGateways(ctx, &ec2.DescribeNatGatewaysInput{NatGatewayIds: []string{id}})
if err != nil { /* verify ID/region/permissions before the real operation */ }

Try / catch

response, err := cloud.EC2().DescribeNatGateways(ctx, request)
if err != nil {
  var re *awshttp.ResponseError
  if errors.As(err, &re) && isThrottling(re) { backoff(); continue }
  return nil, fmt.Errorf("error listing Nat Gateways %w", err)
}

Prevention

When it happens

Trigger: DescribeNatGateways with NatGatewayIds=[e.ID] errors: AccessDenied on ec2:DescribeNatGateways, throttling, invalid NAT gateway ID format, or network failure.

Common situations: State store references a NAT gateway from another region/account; IAM policy missing describe permission; transient AWS throttling in CI loops.

Related errors


AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/a8dbc13df28c368b. Report an issue: GitHub.