{"record":{"id":"5c319b435d76d000","repo":"GoogleContainerTools/skaffold","slug":"error-in-v-job-execution-event-type-v","errorCode":null,"errorMessage":"error in %v job execution, event type: %v","messagePattern":"error in (.+?) job execution, event type: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/skaffold/actions/k8sjob/task.go","lineNumber":213,"sourceCode":"\t})\n\n\treturn g.Wait()\n}\n\nfunc (t Task) watchJob(ctx context.Context, jobManifest batchv1.Job, jobsManager typesbatchv1.JobInterface) error {\n\twatcher, err := jobsManager.Watch(ctx, v1.ListOptions{\n\t\tFieldSelector: fmt.Sprintf(\"metadata.name=%v\", jobManifest.Name),\n\t})\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdefer watcher.Stop()\n\tvar jobErr error\n\t// The ctx is used by the watcher, so if the ctx is canceled, the channel will close finishing the loop.\n\tfor event := range watcher.ResultChan() {\n\t\tif event.Type == apiwatch.Deleted || event.Type == apiwatch.Error {\n\t\t\tjobErr = fmt.Errorf(\"error in %v job execution, event type: %v\", jobManifest.Name, event.Type)\n\t\t\tbreak\n\t\t}\n\n\t\tjobState, ok := event.Object.(*batchv1.Job)\n\t\tif ok && jobState.Status.Failed > 0 {\n\t\t\tjobErr = fmt.Errorf(\"error in %v job execution, job failed\", jobManifest.Name)\n\t\t\tbreak\n\t\t}\n\n\t\tif ok && jobState.Status.Succeeded > 0 {\n\t\t\tbreak\n\t\t}\n\t}\n\n\t// We need this condition to check when the ctx was cancelled due to a timeout. In that case, the previous\n\t// watcher.ResultChan stops without reporting an error.\n\tif ctx.Err() != nil && jobErr == nil {\n\t\tjobErr = ctx.Err()","sourceCodeStart":195,"sourceCodeEnd":231,"githubUrl":"https://github.com/GoogleContainerTools/skaffold/blob/a1189de023efc32d4b8e11f395acc678aa555011/pkg/skaffold/actions/k8sjob/task.go#L195-L231","documentation":"watchJob streams Job events from the API server; if the watched Job is Deleted or the watch emits an Error event, the task records 'error in <job> job execution, event type: <type>' and stops. This indicates the job object disappeared or the watch errored before the job could succeed.","triggerScenarios":"During a k8sjob action run: someone/something deletes the Job (external kubectl delete, namespace teardown, TTL-based GC, garbage collector removing the Job's owner) or the watch connection yields an Error event.","commonSituations":"Manual kubectl delete job during a long test; cluster autoscaler draining the node and GC'ing resources; RBAC/event issues; job manifest path pointing into a namespace being torn down concurrently.","solutions":["Check kubectl get jobs -n <ns> and cluster events (kubectl get events) to see who deleted the Job or why the watch errored.","Re-run the action once cluster conditions stabilize; transient watch errors often disappear on retry.","Ensure nothing concurrently deletes Jobs in the namespace (CI cleanup scripts, TTL controllers ttlSecondsAfterFinished, GC policies)."],"exampleFix":"// before (job spec)\nspec:\n  ttlSecondsAfterFinished: 0  # job deleted immediately -> watch sees Deleted\n// after\nspec:\n  ttlSecondsAfterFinished: 3600","handlingStrategy":"retry","validationCode":"// Ensure no concurrent Job deletions: check TTL and cleanup policies\nkubectl get jobs -n <ns> -o jsonpath='{.items[*].spec.ttlSecondsAfterFinished}'","typeGuard":null,"tryCatchPattern":"try {\n  await skaffold.verify(...);\n} catch (e) {\n  if (/event type: (Deleted|Error)/.test(String(e))) {\n    await new Promise(r => setTimeout(r, 5000)); // transient watch/delete storms\n    await skaffold.verify(...);\n    return;\n  }\n  throw e;\n}","preventionTips":["Do not run kubectl delete job / cleanup scripts while skaffold actions run","Set ttlSecondsAfterFinished high enough to outlive the watch","Keep the job's namespace free of concurrent teardown (CI stage isolation)"],"tags":["kubernetes","jobs","watch-events","skaffold"],"backgroundTag":"k8s-job-deleted-or-watch-error","analyzedSha":"a1189de023efc32d4b8e11f395acc678aa555011","analyzedAt":"2026-09-05T12:09:27.064Z","contentChangedAt":"2026-09-05T12:09:27.064Z","schemaVersion":2},"datasetVersion":"2026-09-12T17:17:11.597Z"}