{"record":{"id":"655e58b800634943","repo":"kubernetes/kops","slug":"failed-to-complete-lifecylce-action-w","errorCode":null,"errorMessage":"failed to complete lifecylce action: %w","messagePattern":"failed to complete lifecylce action: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"upup/pkg/fi/nodeup/command.go","lineNumber":402,"sourceCode":"\n\t// Return rather than exit, so that the retry loop in cmd/nodeup gets to run:\n\t// kops-configuration.service is Type=oneshot, so a bootstrap that exits here is never\n\t// retried and the node never joins the cluster.\n\terr = context.RunTasks(options)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"error running tasks: %w\", err)\n\t}\n\n\terr = target.Finish(taskMap)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"error closing target: %w\", err)\n\t}\n\n\tif nodeupConfig.EnableLifecycleHook {\n\t\tif bootConfig.CloudProvider == api.CloudProviderAWS {\n\t\t\terr := completeWarmingLifecycleAction(ctx, cloud, modelContext)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"failed to complete lifecylce action: %w\", err)\n\t\t\t}\n\t\t}\n\t}\n\treturn nil\n}\n\nfunc getMachineType(ctx context.Context) (string, error) {\n\tconfig, err := awsconfig.LoadDefaultConfig(ctx)\n\tif err != nil {\n\t\treturn \"\", fmt.Errorf(\"failed to load AWS config: %w\", err)\n\t}\n\n\tmetadata := imds.NewFromConfig(config)\n\n\t// Get the actual instance type by querying the EC2 instance metadata service.\n\tresult, err := metadata.GetMetadata(ctx, &imds.GetMetadataInput{\n\t\tPath: \"instance-type\",\n\t})","sourceCodeStart":384,"sourceCodeEnd":420,"githubUrl":"https://github.com/kubernetes/kops/blob/4c8573c808a73d578c5eadc86d410646ea0b0d73/upup/pkg/fi/nodeup/command.go#L384-L420","documentation":"nodeup's Run() wraps any failure from completeWarmingLifecycleAction with this message when EnableLifecycleHook is set on AWS. It means the node finished bootstrapping but could not signal the ASG 'CONTINUE' for the kops-warmpool lifecycle hook, so the instance remains in aws:autoscaling:startup wait state and will eventually time out and be terminated/replaced by the ASG.","triggerScenarios":"nodeupConfig.EnableLifecycleHook==true AND CloudProvider==AWS AND completeWarmingLifecycleAction returns an error (DescribeLifecycleHooks or CompleteLifecycleAction API failure, ASG name mismatch, or missing IAM permission autoscaling:DescribeLifecycleHooks/CompleteLifecycleAction).","commonSituations":"Instance profile lacks autoscaling:CompleteLifecycleAction permission; the ASG name built as <instancegroup>.<cluster> does not match the real ASG; the hook 'kops-warmpool' was not created because the cluster spec lacks the warm pool lifecycle configuration; IMDS/network outage breaking the AWS SDK call.","solutions":["Verify the instance's IAM role grants autoscaling:DescribeLifecycleHooks and autoscaling:CompleteLifecycleAction on the cluster's ASGs.","Confirm the ASG lifecycle hook named 'kops-warmpool' exists on the ASG (aws autoscaling describe-lifecycle-hooks --auto-scaling-group-name <ig>.<cluster>).","Check cluster spec enables warm-pool lifecycle hooks so kOps creates the hook that nodeup expects.","Inspect the wrapped cause (%w) in the nodeup log to distinguish IAM/denied errors from not-found errors, then fix accordingly."],"exampleFix":"// before: no autoscaling permissions on node role\n// after: attach policy to instance profile\n{\n  \"Effect\": \"Allow\",\n  \"Action\": [\"autoscaling:DescribeLifecycleHooks\", \"autoscaling:CompleteLifecycleAction\"],\n  \"Resource\": \"*\"\n}","handlingStrategy":"try-catch","validationCode":"aws autoscaling describe-lifecycle-hooks --auto-scaling-group-name <ig>.<cluster> --lifecycle-hook-names kops-warmpool\naws iam simulate-principal-policy --policy-source-arn <nodeRoleArn> --action-names autoscaling:CompleteLifecycleAction autoscaling:DescribeLifecycleHooks","typeGuard":null,"tryCatchPattern":"err := runNodeup(ctx)\nif err != nil && strings.Contains(err.Error(), \"failed to complete lifecycle\") {\n    // instance will sit in startup-wait; retry nodeup or complete the action manually:\n    // aws autoscaling complete-lifecycle-action --lifecycle-action-result CONTINUE ...\n    return fmt.Errorf(\"lifecycle signal failed, instance will time out in ASG wait state: %w\", err)\n}","preventionTips":["Include autoscaling lifecycle permissions in the kOps node instance profile from day one.","Enable warm-pool lifecycle hooks in the cluster spec so the kops-warmpool hook actually exists.","Keep nodeup version in sync with cluster spec version to avoid naming mismatches.","Set a heartbeat timeout long enough for nodeup to finish before AWS abandons the action."],"tags":["aws","autoscaling","lifecycle-hook","nodeup"],"backgroundTag":"lifecycle-hook-complete-failed","analyzedSha":"4c8573c808a73d578c5eadc86d410646ea0b0d73","analyzedAt":"2026-09-05T04:13:19.212Z","contentChangedAt":"2026-09-05T04:13:19.212Z","schemaVersion":2},"datasetVersion":"2026-09-12T12:17:11.808Z"}