{"record":{"id":"383538ab162fbe7a","repo":"kubernetes/kops","slug":"error-describing-target-health-w","errorCode":null,"errorMessage":"error describing target health: %w","messagePattern":"error describing target health: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"upup/pkg/fi/cloudup/awsup/aws_cloud.go","lineNumber":629,"sourceCode":"\t}\n\n\treturn nil\n}\n\nfunc deregisterInstanceFromTargetGroup(ctx context.Context, c AWSCloud, targetGroupArn string, instanceId string) error {\n\tklog.Infof(\"Deregistering instance from targetGroup: %s\", targetGroupArn)\n\n\tfor {\n\t\tinstanceDraining := false\n\n\t\tresponse, err := c.ELBV2().DescribeTargetHealth(ctx, &elbv2.DescribeTargetHealthInput{\n\t\t\tTargetGroupArn: aws.String(targetGroupArn),\n\t\t\tTargets: []elbv2types.TargetDescription{{\n\t\t\t\tId: aws.String(instanceId),\n\t\t\t}},\n\t\t})\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"error describing target health: %w\", err)\n\t\t}\n\n\t\t// there will be only one target in the DescribeTargetHealth response.\n\t\t// DescribeTargetHealth response will contain a target even if the targetId doesn't exist.\n\t\t// all other states besides TargetHealthStateUnused means that the instance may still be serving traffic.\n\t\tif response.TargetHealthDescriptions[0].TargetHealth.State != elbv2types.TargetHealthStateEnumUnused {\n\t\t\t_, err = c.ELBV2().DeregisterTargets(ctx, &elbv2.DeregisterTargetsInput{\n\t\t\t\tTargetGroupArn: aws.String(targetGroupArn),\n\t\t\t\tTargets: []elbv2types.TargetDescription{{\n\t\t\t\t\tId: aws.String(instanceId),\n\t\t\t\t}},\n\t\t\t})\n\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"error deregistering target: %w\", err)\n\t\t\t}\n\n\t\t\tinstanceDraining = true","sourceCodeStart":611,"sourceCodeEnd":647,"githubUrl":"https://github.com/kubernetes/kops/blob/4c8573c808a73d578c5eadc86d410646ea0b0d73/upup/pkg/fi/cloudup/awsup/aws_cloud.go#L611-L647","documentation":"deregisterInstanceFromTargetGroup polls DescribeTargetHealth to determine whether the instance target is still serving traffic; an AWS SDK error from that call aborts with this wrapped error. Since the drain loop cannot know the target's state, it stops and the instance is not terminated.","triggerScenarios":"c.ELBV2().DescribeTargetHealth returns an error during the drain polling loop: AccessDenied (missing elasticloadbalancing:DescribeTargetHealth), TargetGroupNotFound (TG deleted while still on the ASG), throttling, or network failure.","commonSituations":"Target group removed out-of-band but still listed in the ASG's TargetGroupARNs; IAM gaps; API throttling; cross-region ARN mismatch where the TG lives in a different region than the client.","solutions":["Check the wrapped AWS error: AccessDenied → fix IAM; TargetGroupNotFound → remove the stale TG from the ASG","Ensure IAM policy includes elasticloadbalancing:DescribeTargetHealth","Verify kOps client region matches the target group's region","Retry on throttling/transient errors"],"exampleFix":"// before\n// error describing target health: ValidationError: Target group 'arn:aws:elasticloadbalancing:...' not found\n// after: detach stale TG from ASG then retry\naws autoscaling detach-load-balancer-target-groups --auto-scaling-group-name nodes --target-group-arns arn:aws:elasticloadbalancing:...:stale-tg","handlingStrategy":"try-catch","validationCode":"out, err := elbv2Svc.DescribeTargetGroups(&elbv2.DescribeTargetGroupsInput{TargetGroupArns: []string{tgArn}})\nif err != nil || len(out.TargetGroups) == 0 {\n\treturn fmt.Errorf(\"target group %s not found in client region before drain\", tgArn)\n}","typeGuard":"func isTargetGroupNotFound(err error) bool {\n\tvar ae smithy.APIError\n\treturn errors.As(err, &ae) && ae.ErrorCode() == \"TargetGroupNotFound\"\n}","tryCatchPattern":"if err := cloud.DeregisterInstance(inst); err != nil {\n\tvar ae smithy.APIError\n\tif errors.As(err, &ae) && ae.ErrorCode() == \"TargetGroupNotFound\" {\n\t\treturn detachStaleTargetGroup(asgName, tgArn) // then retry drain\n\t}\n\treturn err\n}","preventionTips":["Grant elasticloadbalancing:DescribeTargetHealth in IAM","Keep the kOps AWS client region identical to the target group's region","Audit ASG TargetGroupARNs for deleted groups before updates","Handle throttling with backoff in any custom drain automation"],"tags":["aws","elbv2","target-group","health-check"],"backgroundTag":"describe-target-health-failed","analyzedSha":"4c8573c808a73d578c5eadc86d410646ea0b0d73","analyzedAt":"2026-09-05T04:13:19.212Z","contentChangedAt":"2026-09-05T04:13:19.212Z","schemaVersion":2},"datasetVersion":"2026-09-12T07:17:12.445Z"}