{"record":{"id":"84f859f1fed01c81","repo":"hashicorp/nomad","slug":"failed-to-process-eval-v-84f859","errorCode":null,"errorMessage":"failed to process eval: %v","messagePattern":"failed to process eval: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"scheduler/scheduler_system.go","lineNumber":77,"sourceCode":"\n// NewSystemScheduler is a factory function to instantiate a new system\n// scheduler.\nfunc NewSystemScheduler(logger log.Logger, eventsCh chan<- any, state sstructs.State, planner sstructs.Planner) sstructs.Scheduler {\n\treturn &SystemScheduler{\n\t\tlogger:   logger.Named(\"system_sched\"),\n\t\teventsCh: eventsCh,\n\t\tstate:    state,\n\t\tplanner:  planner,\n\t}\n}\n\n// Process is used to handle a single evaluation.\nfunc (s *SystemScheduler) Process(eval *structs.Evaluation) (err error) {\n\n\tdefer func() {\n\t\tif r := recover(); r != nil {\n\t\t\ts.logger.Error(\"processing eval panicked scheduler - please report this as a bug!\", \"eval_id\", eval.ID, \"error\", r, \"stack_trace\", string(debug.Stack()))\n\t\t\terr = fmt.Errorf(\"failed to process eval: %v\", r)\n\t\t}\n\t}()\n\n\t// Store the evaluation\n\ts.eval = eval\n\n\t// Update our logger with the eval's information\n\ts.logger = s.logger.With(\"eval_id\", eval.ID, \"job_id\", eval.JobID, \"namespace\", eval.Namespace)\n\n\t// Verify the evaluation trigger reason is understood\n\tif !s.canHandle(eval.TriggeredBy) {\n\t\tdesc := fmt.Sprintf(\"scheduler cannot handle '%s' evaluation reason\", eval.TriggeredBy)\n\t\treturn setStatus(s.logger, s.planner, s.eval, nil,\n\t\t\ts.failedTGAllocs, s.planAnnotations, structs.EvalStatusFailed, desc,\n\t\t\ts.queuedAllocs, s.deployment.GetID())\n\t}\n\n\tlimit := maxSystemScheduleAttempts","sourceCodeStart":59,"sourceCodeEnd":95,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/scheduler/scheduler_system.go#L59-L95","documentation":"SystemScheduler.Process recovers any panic raised while handling a single evaluation, logs the stack trace, and converts it into this error. Like the sysbatch variant, it indicates an internal scheduler bug rather than a user configuration problem, and Nomad requests a bug report.","triggerScenarios":"Any panic inside the system scheduler during Process — e.g. a nil dereference in the scheduling stack, malformed evaluation input, or a downstream step (placement, plan) panicking.","commonSituations":"Nomad bug triggered by unusual system-job or eval data; partially initialized evaluation passed via custom tooling; regression introduced in a Nomad release.","solutions":["Collect the 'stack_trace' from the server log and report the bug to Nomad with eval ID and version.","Inspect the triggering evaluation and system job spec for malformed or unexpected data.","Retry the evaluation; panics may be data-specific rather than persistent.","Upgrade to the latest Nomad release where the panic may already be patched."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"if eval == nil || eval.ID == \"\" || eval.JobID == \"\" {\n    return fmt.Errorf(\"malformed evaluation for system scheduler\")\n}","typeGuard":"func isValidEval(e *structs.Evaluation) bool {\n    return e != nil && e.ID != \"\" && e.Namespace != \"\" && e.JobID != \"\"\n}","tryCatchPattern":"func safeProcessSystem(s *scheduler.SystemScheduler, eval *structs.Evaluation) (err error) {\n    defer func() {\n        if r := recover(); r != nil {\n            err = fmt.Errorf(\"failed to process eval: %v\", r) // log and report stack trace\n        }\n    }()\n    return s.Process(eval)\n}","preventionTips":["Report panics with stack traces and Nomad version to the project.","Validate system job specs and evaluations before processing.","Upgrade to the latest Nomad release for panic fixes.","Alert on repeated 'processing eval panicked' log lines."],"tags":["nomad","scheduler","panic","system"],"backgroundTag":"scheduler-panic-recovered","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}