GoogleContainerTools/skaffold · error

could not fetch stateful sets: %w

Error message

could not fetch stateful sets: %w

What it means

getStatefulSets wraps any error from client.AppsV1().StatefulSets(ns).List (run-id label selector) with 'could not fetch stateful sets'. Note the slightly different wording from the statusCheck-level wrapper ('statefulsets'). statusCheck reports it as STATUSCHECK_STATEFULSET_FETCH_ERR.

Source

Thrown at pkg/skaffold/kubernetes/status/status_check.go:378

		pd := diag.New([]string{d.Namespace}).
			WithLabel(label.RunIDLabel, l.Labels()[label.RunIDLabel]).
			WithValidators([]validator.Validator{validator.NewPodValidator(client, validator.NewDeploymentPodsSelector(client, d))})

		for k, v := range d.Spec.Template.Labels {
			pd = pd.WithLabel(k, v)
		}

		resources[i] = resource.NewResource(d.Name, resource.ResourceTypes.Deployment, d.Namespace, deadline, tolerateFailures).WithValidator(pd)
	}
	return resources, nil
}

func getStatefulSets(ctx context.Context, client kubernetes.Interface, ns string, l *label.DefaultLabeller, deadline time.Duration, tolerateFailures bool) ([]*resource.Resource, error) {
	sets, err := client.AppsV1().StatefulSets(ns).List(ctx, metav1.ListOptions{
		LabelSelector: l.RunIDSelector(),
	})
	if err != nil {
		return nil, fmt.Errorf("could not fetch stateful sets: %w", err)
	}

	resources := make([]*resource.Resource, len(sets.Items))
	for i, ss := range sets.Items {
		pd := diag.New([]string{ss.Namespace}).
			WithLabel(label.RunIDLabel, l.Labels()[label.RunIDLabel]).
			WithValidators([]validator.Validator{validator.NewPodValidator(client, validator.NewStatefulSetPodsSelector(client, ss))})

		for k, v := range ss.Spec.Template.Labels {
			pd = pd.WithLabel(k, v)
		}

		resources[i] = resource.NewResource(ss.Name, resource.ResourceTypes.StatefulSet, ss.Namespace, deadline, tolerateFailures).WithValidator(pd)
	}
	return resources, nil
}

func pollResourceStatus(ctx context.Context, cfg Config, r *resource.Resource) {

View on GitHub (pinned to a1189de023)

Solutions

  1. Reproduce with 'kubectl get statefulsets -n <ns>' using the same kubeconfig
  2. Grant statefulsets.apps get/list/watch in RBAC if the error is 403
  3. Verify namespace and current context are correct
  4. Restore network/VPN to the control plane and refresh credentials
  5. Retry the status check once the API server responds

Example fix

// before: role without statefulsets access
//   kubectl create role ci --verb=get,list --resource=deployments.apps,pods
// after:
//   kubectl create role ci --verb=get,list,watch --resource=deployments.apps,pods,statefulsets.apps
Defensive patterns

Strategy: retry

Validate before calling

const canList = execSync(`kubectl auth can-i list statefulsets.apps -n ${ns}`).toString().trim();
if (canList !== 'yes') throw new Error(`RBAC: cannot list statefulsets in ${ns}`);
execSync(`kubectl get statefulsets -n ${ns} --request-timeout=10s`);

Type guard

function isStatefulSetsFetchErr(err) {
  return err != null && typeof err.message === 'string' && err.message.includes('could not fetch stateful sets');
}

Try / catch

for (let attempt = 0; attempt < 3; attempt++) {
  try { await statusCheck(); break; }
  catch (err) {
    if (isStatefulSetsFetchErr(err) && /timeout|unreachable|connection/i.test(err.message) && attempt < 2) {
      await sleep(2000 * (attempt + 1));
      continue;
    }
    throw err;
  }
}

Prevention

When it happens

Trigger: StatefulSets List call fails per namespace: API server unreachable, RBAC denial on statefulsets.apps, missing namespace, or timeout against getDeadline.

Common situations: Cluster roles granting deployments but not statefulsets; VPN disconnect during status check; wrong namespace in config; cluster control plane overloaded/unreachable right after deploy.

Related errors


AI-assisted analysis of GoogleContainerTools/skaffold@a1189de023 (2026-09-05). Data as JSON: /api/errors/4457c2462e266b98. Report an issue: GitHub.