kubernetes/kops · error

failed to register instance from targetGroups: %w

Error message

failed to register instance from targetGroups: %w

What it means

deregisterInstanceFromTargetGroups fans out one goroutine per target group ARN to drain the instance; when any of them fails, eg.Wait() aggregates the error and it is wrapped with this message. Despite saying 'register', it is emitted on the deregistration path — it means the instance could not be proven unused/drained in at least one target group before termination.

Source

Thrown at upup/pkg/fi/cloudup/awsup/aws_cloud.go:610

		time.Sleep(5 * time.Second)
	}
	return nil
}

// deregisterInstanceFromTargetGroups ensures that instances are fully unused in the corresponding targetGroups before instance termination.
// this ensures that connections are fully drained from the instance before terminating.
func deregisterInstanceFromTargetGroups(ctx context.Context, c AWSCloud, targetGroupArns []string, instanceId string) error {
	eg, _ := errgroup.WithContext(context.Background())

	for _, targetGroupArn := range targetGroupArns {
		arn := targetGroupArn
		eg.Go(func() error {
			return deregisterInstanceFromTargetGroup(ctx, c, arn, instanceId)
		})
	}

	if err := eg.Wait(); err != nil {
		return fmt.Errorf("failed to register instance from targetGroups: %w", err)
	}

	return nil
}

func deregisterInstanceFromTargetGroup(ctx context.Context, c AWSCloud, targetGroupArn string, instanceId string) error {
	klog.Infof("Deregistering instance from targetGroup: %s", targetGroupArn)

	for {
		instanceDraining := false

		response, err := c.ELBV2().DescribeTargetHealth(ctx, &elbv2.DescribeTargetHealthInput{
			TargetGroupArn: aws.String(targetGroupArn),
			Targets: []elbv2types.TargetDescription{{
				Id: aws.String(instanceId),
			}},
		})
		if err != nil {

View on GitHub (pinned to 4c8573c808)

Solutions

1. Inspect the wrapped inner error(s) to identify the failing target group and underlying AWS error code
2. Verify IAM grants elasticloadbalancing:DescribeTargetHealth and elasticloadbalancing:DeregisterTargets
3. Confirm the target group ARNs still exist; detach stale target groups from the ASG
4. Retry — throttling/transient errors during the 5s polling drain loop often clear

Example fix

// before: message is confusing ('failed to register...')
// failed to register instance from targetGroups: error deregistering target: AccessDenied
// after: read inner cause and fix IAM
{"Effect":"Allow","Action":["elasticloadbalancing:DeregisterTargets","elasticloadbalancing:DescribeTargetHealth"],"Resource":"*"}

for _, arn := range asg.TargetGroupARNs {
	_, err := elbv2Svc.DescribeTargetGroups(&elbv2.DescribeTargetGroupsInput{TargetGroupArns: []string{arn}})
	if err != nil {
		return fmt.Errorf("target group %s unavailable before drain: %w", arn, err)
	}
}

func hasTargetGroups(i *cloudinstances.CloudInstance) bool {
	asg, ok := i.CloudInstanceGroup.Raw.(*autoscalingtypes.AutoScalingGroup)
	return ok && len(asg.TargetGroupARNs) > 0
}

if err := cloud.DeregisterInstance(inst); err != nil {
	if strings.Contains(err.Error(), "targetGroups") {
		// inner cause is DescribeTargetHealth or DeregisterTargets; retry transient ones
		return retryWithBackoff(3, 10*time.Second, func() error { return cloud.DeregisterInstance(inst) })
	}
	return err
}

When it happens

Trigger: One or more deregisterInstanceFromTargetGroup goroutines return an error (DescribeTargetHealth or DeregisterTargets failure) while draining an instance from the ASG's target groups during rolling update/delete.

Common situations: Missing elasticloadbalancing permissions; target group deleted out-of-band; throttling with many target groups; network errors mid-drain.

Related errors

• error deleting TargetGroup %q: %v
• target group not yet created (arn not set)
• creating NLB target group: %w
• modifying NLB target group health check: %w
• error deleting ELB TargetGroup %q: %w

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/742705705627f86a. Report an issue: GitHub.