kubernetes/kops · warning

cluster passed validation %d consecutive times

Error message

cluster passed validation %d consecutive times

What it means

With --wait set, a fully passing validation must occur options.count consecutive times before success. If the wait window expires while the cluster has passed validation fewer than count times (consecutive < count) and --wait is set, the loop returns this error instead of sleeping past the deadline. So the cluster is healthy but not consistently validated for long enough within the allowed time.

Source

Thrown at cmd/kops/validate_cluster.go:238

			if err != nil {
				return nil, fmt.Errorf("unable to marshal JSON: %v", err)
			}
			if _, err := out.Write(j); err != nil {
				return nil, fmt.Errorf("error writing to output: %v", err)
			}
		default:
			return nil, fmt.Errorf("unknown output format: %q", options.output)
		}

		if len(result.Failures) == 0 {
			consecutive++
			if consecutive < options.count {
				klog.Infof("(will retry): cluster passed validation %d consecutive times", consecutive)
				if options.wait > 0 {
					time.Sleep(options.interval)
					continue
				} else {
					return nil, fmt.Errorf("cluster passed validation %d consecutive times", consecutive)
				}
			} else {
				return result, nil
			}
		} else {
			if options.wait > 0 {
				klog.Warningf("(will retry): cluster not yet healthy")
				consecutive = 0
				time.Sleep(options.interval)
				continue
			} else {
				return nil, fmt.Errorf("cluster not yet healthy")
			}
		}
	}
}

func validateClusterOutputTable(result *validation.ValidationCluster, cluster *kopsapi.Cluster, instanceGroups []kopsapi.InstanceGroup, out io.Writer) error {

View on GitHub (pinned to 4c8573c808)

Solutions

1. Increase --wait so count consecutive passes fit: wait >= count * interval plus margin.
2. Reduce --count if you only need a single stable pass.
3. Increase --interval only if the API is rate-limited; otherwise keep it small so more attempts fit in wait.
4. Investigate what intermittently resets consecutive (a failing node or pod) via `kubectl get nodes,pods -A`.
5. Run without --wait when a single validation snapshot is sufficient.

Example fix

// before
kops validate cluster --wait 1m --count 5 --interval 30s
// after
kops validate cluster --wait 10m --count 5 --interval 30s

interval, _ := time.ParseDuration("30s")
count := 3
wait, _ := time.ParseDuration("5m")
if wait < time.Duration(count)*interval {
	return fmt.Errorf("--wait (%s) must exceed count*interval (%s)", wait, time.Duration(count)*interval)
}

err := runValidate(ctx, opts)
if err != nil && strings.Contains(err.Error(), "cluster passed validation") {
	// healthy but not stable long enough: extend wait or lower count
	return runValidate(ctx, withWait(opts, 2*opts.Wait))
}

When it happens

Trigger: `kops validate cluster --wait 1m --count 3 --interval 30s` where validation succeeds twice but the timeout expires before the third consecutive pass.

Common situations: Large --count with small --wait; flaky validation where a transient failure resets consecutive to 0; slow API responses eating the interval budget.

Related errors

• wait time exceeded during validation
• failed to parse apiVersion %q
• failed to find kind in object
• invalid InstanceGroup name: %v
• waiting for kubernetes API to be served: %w

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/e344c11f67477e1e. Report an issue: GitHub.