{"record":{"id":"cd4f825d6f69365b","repo":"GoogleContainerTools/skaffold","slug":"error-in-v-job-execution-job-failed","errorCode":null,"errorMessage":"error in %v job execution, job failed","messagePattern":"error in (.+?) job execution, job failed","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"pkg/skaffold/actions/k8sjob/task.go","lineNumber":219,"sourceCode":"\twatcher, err := jobsManager.Watch(ctx, v1.ListOptions{\n\t\tFieldSelector: fmt.Sprintf(\"metadata.name=%v\", jobManifest.Name),\n\t})\n\tif err != nil {\n\t\treturn err\n\t}\n\n\tdefer watcher.Stop()\n\tvar jobErr error\n\t// The ctx is used by the watcher, so if the ctx is canceled, the channel will close finishing the loop.\n\tfor event := range watcher.ResultChan() {\n\t\tif event.Type == apiwatch.Deleted || event.Type == apiwatch.Error {\n\t\t\tjobErr = fmt.Errorf(\"error in %v job execution, event type: %v\", jobManifest.Name, event.Type)\n\t\t\tbreak\n\t\t}\n\n\t\tjobState, ok := event.Object.(*batchv1.Job)\n\t\tif ok && jobState.Status.Failed > 0 {\n\t\t\tjobErr = fmt.Errorf(\"error in %v job execution, job failed\", jobManifest.Name)\n\t\t\tbreak\n\t\t}\n\n\t\tif ok && jobState.Status.Succeeded > 0 {\n\t\t\tbreak\n\t\t}\n\t}\n\n\t// We need this condition to check when the ctx was cancelled due to a timeout. In that case, the previous\n\t// watcher.ResultChan stops without reporting an error.\n\tif ctx.Err() != nil && jobErr == nil {\n\t\tjobErr = ctx.Err()\n\t\t// Sometimes the timeout reports an error before the job result channel, even though the job already failed.\n\t\t// This is to do a last check and assign the appropriate error.\n\t\tif t.isJobErr(context.TODO(), jobManifest.Name, jobsManager) {\n\t\t\tjobErr = fmt.Errorf(\"error in %v job execution, job failed\", jobManifest.Name)\n\t\t}\n\t}","sourceCodeStart":201,"sourceCodeEnd":237,"githubUrl":"https://github.com/GoogleContainerTools/skaffold/blob/a1189de023efc32d4b8e11f395acc678aa555011/pkg/skaffold/actions/k8sjob/task.go#L201-L237","documentation":"watchJob inspects each Job event object's status; if jobState.Status.Failed > 0, the Job's pod(s) failed and the task records 'error in <job> job execution, job failed'. This is the normal signal that the action's container exited non-zero or its pod was unschedulable/evicted.","triggerScenarios":"A k8sjob action's Job reaches Status.Failed > 0: the action container exits non-zero, hits backOffLimit restarts, or its pod is evicted/OOMKilled during the skaffold run.","commonSituations":"Test command inside the job fails (non-zero exit); insufficient resources so pods get evicted; image pull errors inside the job pod; wrong env/args causing the entrypoint to crash; backoffLimit exhausted after transient node failures.","solutions":["Inspect the failed pod logs: kubectl logs job/<job> -n <ns> (or kubectl describe job) to find the container exit reason, then fix the action's command/args.","Raise resources (limits/requests) or fix OOM/eviction causes in the job manifest's container spec.","Check image availability and pull secrets for the job's namespace if the failure stems from ImagePullBackOff inside the job pod."],"exampleFix":"// before (job container)\nargs: [\"run-tests\", \"--suite\", \"integration\"]  # exit 1 -> job failed\n// after\nargs: [\"run-tests\", \"--suite\", \"unit\"]  # or fix the failing test, verify locally first","handlingStrategy":"try-catch","validationCode":"// Smoke-test the action command locally before running as a Job\ndocker run --rm <action-image> <args...> || echo 'action command fails locally too'","typeGuard":null,"tryCatchPattern":"try {\n  await skaffold.verify(...);\n} catch (e) {\n  if (/job failed/.test(String(e))) {\n    const job = String(e).match(/error in (\\S+) job execution/)?.[1];\n    console.error(`Inspect logs: kubectl logs job/${job} -n <ns>`);\n  }\n  throw e;\n}","preventionTips":["Run the action command in a local container first to validate exit code","Set sane resource requests/limits to avoid OOMKill and evictions","Check imagePullSecrets are present in the target namespace"],"tags":["kubernetes","jobs","pod-failure","skaffold"],"backgroundTag":"k8s-job-failed","analyzedSha":"a1189de023efc32d4b8e11f395acc678aa555011","analyzedAt":"2026-09-05T12:09:27.064Z","contentChangedAt":"2026-09-05T12:09:27.064Z","schemaVersion":2},"datasetVersion":"2026-09-12T17:17:11.597Z"}