kubernetes/kops · error

error reading last kops version used to update: %v

Error message

error reading last kops version used to update: %v

What it means

After parsing the semver marker, Run() also guards read errors: if reading <configBase>/kops-version-updated fails with an error other than ErrNotExist, Run returns this wrapped error. This distinguishes a real storage/permission failure from a simply-missing marker file.

Source

Thrown at upup/pkg/fi/cloudup/apply_cluster.go:297

		if err == nil {
			kopsVersionUpdated := strings.TrimSpace(string(kopsVersionUpdatedBytes))
			version, err := semver.Parse(kopsVersionUpdated)
			if err != nil {
				return nil, fmt.Errorf("error parsing last kops version updated: %v", err)
			}
			if version.GT(semver.MustParse(kopsbase.Version)) {
				fmt.Printf("\n")
				fmt.Printf("%s\n", starline)
				fmt.Printf("\n")
				fmt.Printf("The cluster was last updated by kops version %s\n", kopsVersionUpdated)
				fmt.Printf("To permit updating by the older version %s, run with the --allow-kops-downgrade flag\n", kopsbase.Version)
				fmt.Printf("\n")
				fmt.Printf("%s\n", starline)
				fmt.Printf("\n")
				return nil, fmt.Errorf("kops version older than last used to update the cluster")
			}
		} else if err != os.ErrNotExist {
			return nil, fmt.Errorf("error reading last kops version used to update: %v", err)
		}
	}

	cloud := c.Cloud

	err = validation.DeepValidate(c.Cluster, c.InstanceGroups, true, c.Clientset.VFSContext(), cloud)
	if err != nil {
		return nil, err
	}

	if cluster.Spec.KubernetesVersion == "" {
		return nil, fmt.Errorf("KubernetesVersion not set")
	}
	if cluster.Spec.DNSZone == "" && cluster.PublishesDNSRecords() {
		return nil, fmt.Errorf("DNSZone not set")
	}

	l := &Loader{}

View on GitHub (pinned to 4c8573c808)

Solutions

  1. Fix credentials/permissions so the operator can read objects in the state store bucket (re-authenticate with cloud CLI, reapply IAM/GCS policy granting read on the prefix).
  2. Verify connectivity to the object store from the machine running kops (retry after transient errors).
  3. Confirm the state store path is correct with `kops get clusters --state <base>`; then retry the update.

Example fix

// before (expired credentials)
aws s3 ls s3://state-bucket  # AccessDenied
// after
aws sso login && aws s3 ls s3://state-bucket
Defensive patterns

Strategy: try-catch

Validate before calling

if _, err := configBase.Join(registry.PathKopsVersionUpdated).ReadFile(ctx); err != nil && err != os.ErrNotExist {
    return fmt.Errorf("cannot read state store marker: %w", err)
}

Try / catch

out, err := runKopsUpdate()
if err != nil && strings.Contains(err.Error(), "error reading last kops version used to update") {
    // refresh cloud credentials / check state store perms, then retry once
    refreshCredentials()
    out, err = runKopsUpdate()
}

Prevention

When it happens

Trigger: The VFS ReadFile of registry.PathKopsVersionUpdated fails due to missing S3/GCS/Azure permissions, network failure, object-store API errors, or a broken VFS path — anything other than the object not existing.

Common situations: Revoked or expired cloud credentials on the CI runner; bucket policy changes removing GetObject permission; transient AWS/GCS 5xx or throttling during update; VPN/proxy blocking access to the state store endpoint.

Related errors


AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/794b21c1a08eb2f1. Report an issue: GitHub.