{"record":{"id":"015db1f51802e5c3","repo":"hashicorp/nomad","slug":"failed-to-build-reattachconfig-from-task-state-v","errorCode":null,"errorMessage":"failed to build ReattachConfig from task state: %v","messagePattern":"failed to build ReattachConfig from task state: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"drivers/exec/driver.go","lineNumber":428,"sourceCode":"\t\td.logger.Trace(\"nothing to recover; task already exists\",\n\t\t\t\"task_id\", handle.Config.ID,\n\t\t\t\"task_name\", handle.Config.Name,\n\t\t)\n\t\treturn nil\n\t}\n\n\t// Handle doesn't already exist, try to reattach\n\tvar taskState TaskState\n\tif err := handle.GetDriverState(&taskState); err != nil {\n\t\td.logger.Error(\"failed to decode task state from handle\", \"error\", err, \"task_id\", handle.Config.ID)\n\t\treturn fmt.Errorf(\"failed to decode task state from handle: %v\", err)\n\t}\n\n\t// Create client for reattached executor\n\tplugRC, err := pstructs.ReattachConfigToGoPlugin(taskState.ReattachConfig)\n\tif err != nil {\n\t\td.logger.Error(\"failed to build ReattachConfig from task state\", \"error\", err, \"task_id\", handle.Config.ID)\n\t\treturn fmt.Errorf(\"failed to build ReattachConfig from task state: %v\", err)\n\t}\n\n\texec, pluginClient, err := executor.ReattachToExecutor(\n\t\tplugRC,\n\t\td.logger.With(\"task_name\", handle.Config.Name, \"alloc_id\", handle.Config.AllocID),\n\t\td.compute,\n\t)\n\tif err != nil {\n\t\td.logger.Error(\"failed to reattach to executor\", \"error\", err, \"task_id\", handle.Config.ID)\n\t\treturn fmt.Errorf(\"failed to reattach to executor: %v\", err)\n\t}\n\n\th := &taskHandle{\n\t\texec:         exec,\n\t\tpid:          taskState.Pid,\n\t\tpluginClient: pluginClient,\n\t\ttaskConfig:   taskState.TaskConfig,\n\t\tprocState:    drivers.TaskStateRunning,","sourceCodeStart":410,"sourceCodeEnd":446,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/drivers/exec/driver.go#L410-L446","documentation":"During RecoverTask, the exec driver rebuilds a go-plugin ReattachConfig from the task state persisted in the client state DB and hands it to pstructs.ReattachConfigToGoPlugin. If that conversion fails, Nomad cannot reattach to the still-running executor process and the recovery is aborted. This typically means the persisted ReattachConfig is corrupt, incomplete, or from an incompatible plugin protocol version.","triggerScenarios":"RecoverTask is called after a Nomad client restart with a task whose persisted drivers.TaskState contains a ReattachConfig that fails conversion: nil ReattachConfig, invalid Addr/Pid, or a protocol version go-plugin refuses to parse.","commonSituations":"Corrupted or truncated bolt state DB after a crash; upgrading Nomad across incompatible go-plugin protocol versions so old state entries no longer parse; manually edited or restored client state directory.","solutions":["Inspect the task's persisted state; if the executor process is gone or state is corrupt, stop the task and reschedule it rather than recovering.","Verify the client was not restored from a state directory of a different Nomad version; align versions or wipe the stale state for that allocation.","Check Nomad client logs for the underlying error from ReattachConfigToGoPlugin to confirm which field is invalid.","As a last resort, GC the allocation and let the scheduler place a new one."],"exampleFix":null,"handlingStrategy":"validation","validationCode":"// before relying on recovery, verify the persisted state is usable\nif taskState.ReattachConfig == nil || taskState.ReattachConfig.Pid <= 0 {\n    // treat as unrecoverable: stop + reschedule instead of RecoverTask\n}","typeGuard":"func reattachConfigValid(rc *plugin.ReattachConfig) bool {\n    return rc != nil && rc.Protocol != \"\" && rc.Addr != nil && rc.Pid > 0\n}","tryCatchPattern":null,"preventionTips":["Never restore a Nomad client data dir from a different version's backup.","Pin Nomad versions consistently across upgrades so plugin protocol versions match.","Monitor for client crashes that can truncate the state DB; use filesystem journaling."],"tags":["nomad","exec-driver","plugin-reattach","task-recovery"],"backgroundTag":"plugin-reattach-config-invalid","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}