{"record":{"id":"9812f3ec2e38de82","repo":"hashicorp/nomad","slug":"failed-to-process-eval-v-9812f3","errorCode":null,"errorMessage":"failed to process eval: %v","messagePattern":"failed to process eval: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"scheduler/scheduler_sysbatch.go","lineNumber":68,"sourceCode":"\tplanAnnotations *structs.PlanAnnotations\n}\n\nfunc NewSysBatchScheduler(logger log.Logger, eventsCh chan<- any, state sstructs.State, planner sstructs.Planner) sstructs.Scheduler {\n\treturn &SysBatchScheduler{\n\t\tlogger:   logger.Named(\"sysbatch_sched\"),\n\t\teventsCh: eventsCh,\n\t\tstate:    state,\n\t\tplanner:  planner,\n\t}\n}\n\n// Process is used to handle a single evaluation.\nfunc (s *SysBatchScheduler) Process(eval *structs.Evaluation) (err error) {\n\n\tdefer func() {\n\t\tif r := recover(); r != nil {\n\t\t\ts.logger.Error(\"processing eval panicked scheduler - please report this as a bug!\", \"eval_id\", eval.ID, \"error\", r, \"stack_trace\", string(debug.Stack()))\n\t\t\terr = fmt.Errorf(\"failed to process eval: %v\", r)\n\t\t}\n\t}()\n\n\t// Store the evaluation\n\ts.eval = eval\n\n\t// Update our logger with the eval's information\n\ts.logger = s.logger.With(\"eval_id\", eval.ID, \"job_id\", eval.JobID, \"namespace\", eval.Namespace)\n\n\t// Verify the evaluation trigger reason is understood\n\tif !s.canHandle(eval.TriggeredBy) {\n\t\tdesc := fmt.Sprintf(\"scheduler cannot handle '%s' evaluation reason\", eval.TriggeredBy)\n\t\treturn setStatus(s.logger, s.planner, s.eval, nil,\n\t\t\ts.failedTGAllocs, s.planAnnotations, structs.EvalStatusFailed, desc,\n\t\t\ts.queuedAllocs, \"\")\n\t}\n\n\tlimit := maxSysBatchScheduleAttempts","sourceCodeStart":50,"sourceCodeEnd":86,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/scheduler/scheduler_sysbatch.go#L50-L86","documentation":"SysBatchScheduler.Process wraps any panic raised while handling a single evaluation, logs the stack trace, and converts it into this error. It signals an internal scheduler bug (nil dereference, index-out-of-range, etc.) rather than a user-caused condition. Nomad explicitly asks users to report it as a bug.","triggerScenarios":"Any panic inside the sysbatch scheduler during Process — e.g. nil job/plan fields, malformed evaluation passed to Process, or a bug in a downstream scheduler step that panics.","commonSituations":"A nil or partially initialized *structs.Evaluation passed by custom tooling; Nomad bug triggered by unusual job/eval data; regression after a Nomad upgrade.","solutions":["Capture the 'stack_trace' from the server log and file a bug report with Nomad including eval ID and Nomad version.","Inspect the evaluation that triggered the panic (nomad eval status / state store) for malformed data.","Retry the evaluation; transient panics on one eval may not recur.","Upgrade to the latest Nomad version where the panic may already be fixed."],"exampleFix":"// caller-side handling\n// before\nerr := sysBatch.Process(eval)\n// after\nif err := recoverPanic(func() error { return sysBatch.Process(eval) }); err != nil {\n    logger.Error(\"scheduler panicked\", \"err\", err) // send stack trace upstream\n}","handlingStrategy":"try-catch","validationCode":"if eval == nil || eval.ID == \"\" || eval.JobID == \"\" {\n    return fmt.Errorf(\"malformed evaluation: missing id/job fields\")\n}","typeGuard":"func isValidEval(e *structs.Evaluation) bool {\n    return e != nil && e.ID != \"\" && e.Namespace != \"\" && e.JobID != \"\"\n}","tryCatchPattern":"func safeProcess(s *scheduler.SysBatchScheduler, eval *structs.Evaluation) (err error) {\n    defer func() {\n        if r := recover(); r != nil {\n            err = fmt.Errorf(\"failed to process eval: %v\", r) // report stack trace upstream\n        }\n    }()\n    return s.Process(eval)\n}","preventionTips":["Report panics with stack traces to the Nomad project.","Ensure evaluations are well-formed before passing to Process.","Run the latest Nomad patch release to pick up panic fixes.","Watch server logs for repeated panics on the same eval/job."],"tags":["nomad","scheduler","panic","sysbatch"],"backgroundTag":"scheduler-panic-recovered","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}