derailed/k9s · warning

%d pods failed

Error message

%d pods failed

What it means

Job.diagnose splits the ready column '<succeeded>/<desired>'; when the counts differ AND status.failed > 0 it reports 'N pods failed'. It surfaces batch Jobs whose pods have actually failed (not merely still running) — backoff/restart territory.

Source

Thrown at internal/render/job.go:91

		job.Namespace,
		job.Name,
		computeVulScore(job.Namespace, job.Labels, &job.Spec.Template.Spec),
		ready,
		toDuration(&job.Status),
		jobSelector(&job.Spec),
		cc,
		ii,
		AsStatus(j.diagnose(ready, &job.Status)),
		ToAge(job.GetCreationTimestamp()),
	}

	return nil
}

func (Job) diagnose(ready string, status *batchv1.JobStatus) error {
	tokens := strings.Split(ready, "/")
	if tokens[0] != tokens[1] && status.Failed > 0 {
		return fmt.Errorf("%d pods failed", status.Failed)
	}
	return nil
}

// ----------------------------------------------------------------------------
// Helpers...

const maxShow = 2

func toContainers(p *v1.PodSpec) (containers, images string) {
	cc, ii := parseContainers(p.InitContainers)
	cn, ci := parseContainers(p.Containers)

	cc, ii = append(cc, cn...), append(ii, ci...)

	// Limit to 2 of each...
	if len(cc) > maxShow {
		cc = append(cc[:2], "(+"+strconv.Itoa(len(cc)-maxShow)+")...")

View on GitHub (pinned to 2d3ccc6ba2)

Solutions

1. kubectl describe job <name> and read the events (BackoffLimitExceeded etc.)
2. Fetch failed pod logs: kubectl logs <pod> --previous to see the actual error
3. Fix the root cause (env, command, limits) and re-run: kubectl delete job <name> && kubectl apply -f job.yaml
4. Raise backoffLimit / set restartPolicy: OnFailure only if failures are genuinely transient

Example fix

# before: app needs DB URI, pods fail instantly
containers:
- name: migrate
  image: app:v1
# after
containers:
- name: migrate
  image: app:v1
  env:
  - name: DATABASE_URL
    value: postgres://...

if job.Status.Failed == 0 {
	// no failed pods; skip failure handling
}

Consume the error as status text; on display, trigger log retrieval for failed pods rather than retrying the Job blindly.

When it happens

Trigger: Job pods exiting non-zero (app bug, bad args, missing config); image pull failures recorded as pod failures; completions > parallelism mid-run with early failures; backoffLimit not yet reached.

Common situations: One-shot migrate/batch jobs with wrong env vars; OOMKilled workers (memory limits too low); Jobs racing dependencies (DB not up); cronjobs hitting transient API errors.

Related errors

• expecting Job resource
• expecting a job resource
• no valid selector found for job: %s
• desiring %d replicas got %d available
• desiring %d replicas but %d ready

AI-assisted analysis of derailed/k9s@2d3ccc6ba2 (2026-08-15). Data as JSON: /api/errors/0f95d5df5eefa8dc. Report an issue: GitHub.