kubernetes/kops · critical

failed to get machine type: %w

Error message

failed to get machine type: %w

What it means

After fetching the instance-id, the AWS branch of Run() calls getMachineType(ctx), which loads an AWS SDK config and queries the EC2 Instance Metadata Service for instance-type. Any failure there (config load failure or IMDS request failure) is wrapped as "failed to get machine type". nodeup needs the machine type to pick architecture-specific images and GPU support, so bootstrap stops.

Source

Thrown at upup/pkg/fi/nodeup/command.go:266

	switch bootConfig.CloudProvider {
	case api.CloudProviderAWS:
		instanceIDBytes, err := vfs.Context.ReadFile("metadata://aws/meta-data/instance-id")
		if err != nil {
			return fmt.Errorf("error reading instance-id from AWS metadata: %v", err)
		}
		modelContext.InstanceID = string(instanceIDBytes)

		// Check if WarmPool is enabled first, to avoid additional API calls
		if len(modelContext.NodeupConfig.WarmPoolImages) > 0 {
			modelContext.ConfigurationMode, err = getAWSConfigurationMode(ctx, modelContext)
			if err != nil {
				return err
			}
		}

		modelContext.MachineType, err = getMachineType(ctx)
		if err != nil {
			return fmt.Errorf("failed to get machine type: %w", err)
		}

		// If Nvidia is enabled in the cluster, check if this instance has support for it.
		nvidia := modelContext.NodeupConfig.ContainerdConfig.NvidiaGPU
		if nvidia != nil && fi.ValueOf(nvidia.Enabled) {
			// Get the instance type's detailed information.
			instanceType, err := cloud.GetMachineTypeInfo(ctx, ec2types.InstanceType(modelContext.MachineType))
			if err != nil {
				return err
			}

			if instanceType.GPU {
				klog.Info("instance supports GPU acceleration")
				modelContext.GPUVendor = architectures.GPUVendorNvidia
			}
		}
	case api.CloudProviderOpenstack:
		// NvidiaGPU possible to enable only in instance group level in OpenStack. When we assume that GPU is supported

View on GitHub (pinned to 4c8573c808)

Solutions

  1. Test IMDS directly: curl -s -H 'X-aws-ec2-metadata-token: <token>' http://169.254.169.254/latest/meta-data/instance-type
  2. Raise the IMDSv2 hop limit to 2 if nodeup runs in a container (aws ec2 modify-instance-metadata-options --http-put-response-hop-limit 2)
  3. Ensure the node has a region resolvable (AWS_REGION env or instance-level default) so LoadDefaultConfig succeeds
  4. Confirm instance metadata options have HttpEnabled=true

Example fix

// before: hop-limit 1 blocks SDK IMDSv2 calls from a container
// after
aws ec2 modify-instance-metadata-options --instance-id i-0abc123 \
  --http-put-response-hop-limit 2 --http-tokens required
Defensive patterns

Strategy: retry

Validate before calling

TOKEN=$(curl -sX PUT 'http://169.254.169.254/latest/api/token' -H 'X-aws-ec2-metadata-token-ttl-seconds: 60') && curl -s -H "X-aws-ec2-metadata-token: $TOKEN" http://169.254.169.254/latest/meta-data/instance-type

Try / catch

if err := nodeupCmd.Run(ctx); err != nil {
    var retryable = strings.Contains(err.Error(), "failed to get machine type")
    if retryable {
        return backoffRetry(nodeupCmd.Run, 3)
    }
    return err
}

Prevention

When it happens

Trigger: getMachineType(ctx) fails: awsconfig.LoadDefaultConfig errors (no credentials/region resolution) or imds.GetMetadata("instance-type") errors (IMDS unreachable, IMDSv2 token issues, metadata disabled).

Common situations: Node running with IMDS hop-limit 1 inside a container; IMDS disabled via instance metadata options; SDK unable to resolve a region because no region is configured anywhere; running nodeup off-EC2 with the AWS provider.

Related errors


AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/7949b3a7016bd3dd. Report an issue: GitHub.