kubernetes/kops · error

failed to register instance from targetGroups

Error message

failed to register instance from targetGroups: %w

What it means

deregisterInstanceFromTargetGroups fans out one goroutine per target group ARN to drain the instance; when any of them fails, eg.Wait() aggregates the error and it is wrapped with this message. Despite saying 'register', it is emitted on the deregistration path — it means the instance could not be proven unused/drained in at least one target group before termination.

Solutions

1. Inspect the wrapped inner error(s) to identify the failing target group and underlying AWS error code
2. Verify IAM grants elasticloadbalancing:DescribeTargetHealth and elasticloadbalancing:DeregisterTargets
3. Confirm the target group ARNs still exist; detach stale target groups from the ASG
4. Retry — throttling/transient errors during the 5s polling drain loop often clear

Example fix

// before: message is confusing ('failed to register...')
// failed to register instance from targetGroups: error deregistering target: AccessDenied
// after: read inner cause and fix IAM
{"Effect":"Allow","Action":["elasticloadbalancing:DeregisterTargets","elasticloadbalancing:DescribeTargetHealth"],"Resource":"*"}

for _, arn := range asg.TargetGroupARNs {
	_, err := elbv2Svc.DescribeTargetGroups(&elbv2.DescribeTargetGroupsInput{TargetGroupArns: []string{arn}})
	if err != nil {
		return fmt.Errorf("target group %s unavailable before drain: %w", arn, err)
	}
}

func hasTargetGroups(i *cloudinstances.CloudInstance) bool {
	asg, ok := i.CloudInstanceGroup.Raw.(*autoscalingtypes.AutoScalingGroup)
	return ok && len(asg.TargetGroupARNs) > 0
}

if err := cloud.DeregisterInstance(inst); err != nil {
	if strings.Contains(err.Error(), "targetGroups") {
		// inner cause is DescribeTargetHealth or DeregisterTargets; retry transient ones
		return retryWithBackoff(3, 10*time.Second, func() error { return cloud.DeregisterInstance(inst) })
	}
	return err
}

When it happens

Trigger: One or more deregisterInstanceFromTargetGroup goroutines return an error (DescribeTargetHealth or DeregisterTargets failure) while draining an instance from the ASG's target groups during rolling update/delete.

Common situations: Missing elasticloadbalancing permissions; target group deleted out-of-band; throttling with many target groups; network errors mid-drain.

Related errors

• creating NLB target group
• error deleting ELB TargetGroup
• error deleting TargetGroup
• error deregistering target
• error describing target health

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/742705705627f86a. Report an issue: GitHub.

Appendix: source

Thrown at upup/pkg/fi/cloudup/awsup/aws_cloud.go:610

		time.Sleep(5 * time.Second)
	}
	return nil
}

// deregisterInstanceFromTargetGroups ensures that instances are fully unused in the corresponding targetGroups before instance termination.
// this ensures that connections are fully drained from the instance before terminating.
func deregisterInstanceFromTargetGroups(ctx context.Context, c AWSCloud, targetGroupArns []string, instanceId string) error {
	eg, _ := errgroup.WithContext(context.Background())

	for _, targetGroupArn := range targetGroupArns {
		arn := targetGroupArn
		eg.Go(func() error {
			return deregisterInstanceFromTargetGroup(ctx, c, arn, instanceId)
		})
	}

	if err := eg.Wait(); err != nil {
		return fmt.Errorf("failed to register instance from targetGroups: %w", err)
	}

	return nil
}

func deregisterInstanceFromTargetGroup(ctx context.Context, c AWSCloud, targetGroupArn string, instanceId string) error {
	klog.Infof("Deregistering instance from targetGroup: %s", targetGroupArn)

	for {
		instanceDraining := false

		response, err := c.ELBV2().DescribeTargetHealth(ctx, &elbv2.DescribeTargetHealthInput{
			TargetGroupArn: aws.String(targetGroupArn),
			Targets: []elbv2types.TargetDescription{{
				Id: aws.String(instanceId),
			}},
		})
		if err != nil {

View on GitHub (pinned to 4c8573c808)