{"record":{"id":"a0f3b39b8fe1b192","repo":"hashicorp/nomad","slug":"failed-to-initialize-driver-after-it-exited-unexpe","errorCode":null,"errorMessage":"failed to initialize driver after it exited unexpectedly: %v","messagePattern":"failed to initialize driver after it exited unexpectedly: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/allocrunner/taskrunner/task_runner.go","lineNumber":976,"sourceCode":"\tif tr.getDriverHandle() != nil {\n\t\t// Ensure running state is persisted but do *not* append a new\n\t\t// task event as restoring is a client event and not relevant\n\t\t// to a task's lifecycle.\n\t\tif err := tr.updateStateImpl(structs.TaskStateRunning); err != nil {\n\t\t\t//TODO return error and destroy task to avoid an orphaned task?\n\t\t\ttr.logger.Warn(\"error persisting task state\", \"error\", err)\n\t\t}\n\t\treturn nil\n\t}\n\n\t// Start the job if there's no existing handle (or if RecoverTask failed)\n\thandle, net, err := tr.driver.StartTask(taskConfig)\n\tif err != nil {\n\t\t// The plugin has died, try relaunching it\n\t\tif err == bstructs.ErrPluginShutdown {\n\t\t\ttr.logger.Info(\"failed to start task because plugin shutdown unexpectedly; attempting to recover\")\n\t\t\tif err := tr.initDriver(); err != nil {\n\t\t\t\ttaskErr := fmt.Errorf(\"failed to initialize driver after it exited unexpectedly: %v\", err)\n\t\t\t\ttr.EmitEvent(structs.NewTaskEvent(structs.TaskDriverFailure).SetDriverError(taskErr))\n\t\t\t\treturn taskErr\n\t\t\t}\n\n\t\t\thandle, net, err = tr.driver.StartTask(taskConfig)\n\t\t\tif err != nil {\n\t\t\t\ttaskErr := fmt.Errorf(\"failed to start task after driver exited unexpectedly: %v\", err)\n\t\t\t\ttr.EmitEvent(structs.NewTaskEvent(structs.TaskDriverFailure).SetDriverError(taskErr))\n\t\t\t\treturn taskErr\n\t\t\t}\n\t\t} else {\n\t\t\t// Do *NOT* wrap the error here without maintaining whether or not is Recoverable.\n\t\t\t// You must emit a task event failure to be considered Recoverable\n\t\t\ttr.EmitEvent(structs.NewTaskEvent(structs.TaskDriverFailure).SetDriverError(err))\n\t\t\treturn err\n\t\t}\n\t}\n","sourceCodeStart":958,"sourceCodeEnd":994,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/allocrunner/taskrunner/task_runner.go#L958-L994","documentation":"In runDriver, when StartTask fails because the driver plugin shut down (ErrPluginShutdown), Nomad reinitializes the plugin via tr.initDriver; if that reinit fails, the task is marked TaskDriverFailure with this error and aborts.","triggerScenarios":"Thrown at client/allocrunner/taskrunner/task_runner.go:976 when the library encounters an invalid state.","commonSituations":"See trigger scenarios.","solutions":["Inspect client logs for why the driver plugin crashed and failed to reinit","Check driver plugin binaries and dependencies (e.g. docker socket)","Restart the Nomad client to relaunch driver plugins","Reschedule the task once the driver is healthy"],"exampleFix":null,"handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":null,"preventionTips":[],"tags":[],"backgroundTag":null,"analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}