GoogleContainerTools/skaffold · error

attempting to watch verify job in cluster

Error message

attempting to watch verify job in cluster

What it means

watchJob failed to establish a Watch on the batch/v1 Job in the cluster. If the watch API call errors, the verify test is marked failed with 'attempting to watch verify job in cluster' wrapping the kube client error.

Source

Thrown at pkg/skaffold/verify/k8sjob/verify.go:259

	case execErr = <-execCh:
	case <-v.timeout(timeoutDuration):
		execErr = errors.New(fmt.Sprintf("%q running k8s job timed out after : %v", tc.Name, *timeoutDuration))
		v.logger.CancelJobLogger(job.Name)
		if err := k8sjobutil.ForceJobDelete(ctx, job.Name, clientset.BatchV1().Jobs(job.Namespace), &v.kubectl); err != nil {
			execErr = errors.Wrap(execErr, err.Error())
		}
		eventV2.VerifyFailed(tc.Name, execErr)
	}

	return execErr
}

func (v *Verifier) watchJob(ctx context.Context, clientset k8sclient.Interface, job *batchv1.Job, tc latest.VerifyTestCase) error {
	w, err := clientset.BatchV1().Jobs(job.Namespace).Watch(ctx,
		metav1.ListOptions{FieldSelector: fmt.Sprintf("metadata.name=%s", job.Name)})
	if err != nil {
		eventV2.VerifyFailed(tc.Name, err)
		return errors.Wrap(err, "attempting to watch verify job in cluster")
	}
	defer w.Stop()

	w, err = clientset.CoreV1().Pods(job.Namespace).Watch(ctx,
		metav1.ListOptions{
			LabelSelector: labels.Set(map[string]string{"job-name": job.Name}).String(),
		})
	if err != nil {
		eventV2.VerifyFailed(tc.Name, err)
		return errors.Wrap(err, "attempting to watch verify pods in cluster")
	}
	defer w.Stop()

	var podErr error
	for event := range w.ResultChan() {
		pod, ok := event.Object.(*corev1.Pod)
		if ok {
			if pod.Status.Phase == corev1.PodSucceeded {

View on GitHub (pinned to a1189de023)

Solutions

1. Verify RBAC: `kubectl auth can-i watch jobs -n <ns>` and add the missing role rules
2. Confirm the job/namespace exist: `kubectl get jobs -n <ns>`
3. Check kubeconfig/cluster connectivity and API server health
4. Retry `skaffold verify`; if transient, the watch usually succeeds on rerun

Example fix

// before: role without watch
rules:
- apiGroups: ["batch"]
  resources: ["jobs"]
  verbs: ["create"]
// after
rules:
- apiGroups: ["batch"]
  resources: ["jobs"]
  verbs: ["create", "get", "list", "watch"]

// Pre-check watch permission on jobs
if err := exec.Command("kubectl", "auth", "can-i", "watch", "jobs", "-n", ns).Run(); err != nil {
    return fmt.Errorf("serviceaccount cannot watch jobs in namespace %s", ns)
}

if err := verifier.Verify(ctx, out, tc); err != nil {
    if strings.Contains(err.Error(), "attempting to watch verify job in cluster") {
        return fmt.Errorf("job watch failed (add jobs/watch RBAC): %w", err)
    }
    return err
}

When it happens

Trigger: clientset.BatchV1().Jobs(ns).Watch(...) returns an error — RBAC denies watch on jobs, namespace doesn't exist, or API server connectivity fails, right after the job was created.

Common situations: ServiceAccount can create but not watch Jobs (missing 'get/watch jobs' RBAC); API server momentarily unavailable; wrong namespace context; network policies blocking watch streams.

Related errors

• attempting to watch verify pods in cluster
• initializing deployment watcher: %s
• initializing pod watcher for %q: %w
• creating verify job in cluster
• %q running k8s job timed out after : %v

AI-assisted analysis of GoogleContainerTools/skaffold@a1189de023 (2026-09-05). Data as JSON: /api/errors/0da88775af503bf1. Report an issue: GitHub.