{"record":{"id":"c4ff2b0a8212ccdb","repo":"hashicorp/nomad","slug":"failed-to-build-reattachconfig-from-task-state-v-c4ff2b","errorCode":null,"errorMessage":"failed to build ReattachConfig from task state: %v","messagePattern":"failed to build ReattachConfig from task state: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"drivers/rawexec/driver.go","lineNumber":338,"sourceCode":"\tif _, ok := d.tasks.Get(handle.Config.ID); ok {\n\t\td.logger.Trace(\"nothing to recover; task already exists\",\n\t\t\t\"task_id\", handle.Config.ID,\n\t\t\t\"task_name\", handle.Config.Name,\n\t\t)\n\t\treturn nil\n\t}\n\n\t// Handle doesn't already exist, try to reattach\n\tvar taskState TaskState\n\tif err := handle.GetDriverState(&taskState); err != nil {\n\t\td.logger.Error(\"failed to decode task state from handle\", \"error\", err, \"task_id\", handle.Config.ID)\n\t\treturn fmt.Errorf(\"failed to decode task state from handle: %v\", err)\n\t}\n\n\tplugRC, err := pstructs.ReattachConfigToGoPlugin(taskState.ReattachConfig)\n\tif err != nil {\n\t\td.logger.Error(\"failed to build ReattachConfig from task state\", \"error\", err, \"task_id\", handle.Config.ID)\n\t\treturn fmt.Errorf(\"failed to build ReattachConfig from task state: %v\", err)\n\t}\n\n\t// Create client for reattached executor\n\texec, pluginClient, err := executor.ReattachToExecutor(\n\t\tplugRC,\n\t\td.logger.With(\"task_name\", handle.Config.Name, \"alloc_id\", handle.Config.AllocID),\n\t\td.compute,\n\t)\n\tif err != nil {\n\t\td.logger.Error(\"failed to reattach to executor\", \"error\", err, \"task_id\", handle.Config.ID)\n\t\treturn fmt.Errorf(\"failed to reattach to executor: %v\", err)\n\t}\n\n\th := &taskHandle{\n\t\texec:         exec,\n\t\tpid:          taskState.Pid,\n\t\tpluginClient: pluginClient,\n\t\ttaskConfig:   taskState.TaskConfig,","sourceCodeStart":320,"sourceCodeEnd":356,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/drivers/rawexec/driver.go#L320-L356","documentation":"RecoverTask failed while converting the persisted plugin ReattachConfig (gRPC address/pid) into a go-plugin client config via pstructs.ReattachConfigToGoPlugin. The stored ReattachConfig is missing required fields (e.g. empty address or invalid protocol), so the driver cannot even attempt to reconnect to the executor plugin.","triggerScenarios":"Calling RecoverTask where taskState.ReattachConfig decoded but is invalid: empty Addr/Protocol, nil ReattachConfig, or a config written in a format the current Nomad version cannot convert.","commonSituations":"Version skew after upgrade/downgrade of Nomad clients; partially written driver state after a crash; tasks whose executor plugin state was never fully persisted.","solutions":["Check Nomad version consistency across the cluster; reattach config formats changed between releases","Stop and reschedule the allocation (nomad alloc stop <alloc_id>) so a new executor is launched","Inspect the handle state in the client data_dir for empty/invalid reattach fields; treat as corrupt and clear it","Restart the nomad client after cleaning the corrupt state entry"],"exampleFix":null,"handlingStrategy":"fallback","validationCode":null,"typeGuard":"// validate reattach config before conversion\nfunc validReattach(rc *pstructs.ReattachConfig) bool {\n  return rc != nil && rc.Pid > 0 && rc.Addr != nil && rc.Addr.String() != \"\" && rc.Protocol != \"\"\n}","tryCatchPattern":"if err := d.RecoverTask(handle); err != nil {\n  if strings.Contains(err.Error(), \"failed to build ReattachConfig\") {\n    // abandon recovery; let scheduler place a new allocation\n    return fmt.Errorf(\"unrecoverable handle: %w\", err)\n  }\n}","preventionTips":["Avoid restoring client data_dir snapshots across different Nomad versions","Keep cluster agents on one Nomad release","Let Nomad manage handles; never transplant state between hosts","After crashes, prefer nomad alloc stop over manual state surgery"],"tags":["nomad","rawexec","plugin","reattach","task-recovery"],"backgroundTag":"reattach-config-invalid","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}