{"record":{"id":"2019aac2117232e7","repo":"hashicorp/nomad","slug":"failed-to-create-executor-v-2019aa","errorCode":null,"errorMessage":"failed to create executor: %v","messagePattern":"failed to create executor: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"drivers/java/driver.go","lineNumber":495,"sourceCode":"\t\tif err != nil {\n\t\t\treturn nil, nil, fmt.Errorf(\"failed to build mount for resolv.conf: %v\", err)\n\t\t}\n\t\tcfg.Mounts = append(cfg.Mounts, dnsMount)\n\t}\n\n\tcaps, err := capabilities.Calculate(\n\t\tcapabilities.NomadDefaults(), d.config.AllowCaps, driverConfig.CapAdd, driverConfig.CapDrop,\n\t)\n\tif err != nil {\n\t\treturn nil, nil, err\n\t}\n\td.logger.Debug(\"task capabilities\", \"capabilities\", caps)\n\n\texec, pluginClient, err := executor.CreateExecutor(\n\t\td.logger.With(\"task_name\", handle.Config.Name, \"alloc_id\", handle.Config.AllocID),\n\t\td.nomadConfig, executorConfig)\n\tif err != nil {\n\t\treturn nil, nil, fmt.Errorf(\"failed to create executor: %v\", err)\n\t}\n\t// prevent leaking executor in error scenarios\n\tdefer func() {\n\t\tif err != nil {\n\t\t\tpluginClient.Kill()\n\t\t}\n\t}()\n\n\texecCmd := &executor.ExecCommand{\n\t\tCmd:              absPath,\n\t\tArgs:             args,\n\t\tEnv:              cfg.EnvList(),\n\t\tUser:             user,\n\t\tResourceLimits:   true,\n\t\tResources:        cfg.Resources,\n\t\tTaskDir:          cfg.TaskDir().Dir,\n\t\tWorkDir:          driverConfig.WorkDir,\n\t\tStdoutPath:       cfg.StdoutPath,","sourceCodeStart":477,"sourceCodeEnd":513,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/drivers/java/driver.go#L477-L513","documentation":"StartTask creates a go-plugin executor subprocess (executor.CreateExecutor) that will supervise the java process. If the executor plugin cannot be created — plugin binary missing, handshake failure, bad nomadConfig/executorConfig — the error is wrapped with this message. The deferred pluginClient.Kill() prevents leaking the executor on later failures.","triggerScenarios":"Calling StartTask when executor.CreateExecutor fails: the nomad executor plugin binary is absent/mismatched, the plugin handshake times out, executorConfig (LogFile, FSIsolation, Compute) is invalid, or the executor log file path can't be used.","commonSituations":"Nomad version skew between client and executor plugin; corrupted or replaced nomad installation; running in restricted containers where spawning the plugin process or writing executor.out fails; FSIsolation mismatch (chroot on hosts without the chroot content).","solutions":["Restart the Nomad client to restore the executor plugin (re-extract/re-hash plugins).","Ensure the Nomad client installation is intact and all versions match (no partially upgraded binaries).","Check that the task dir is writable so executor.out can be created.","Inspect client logs at debug level for the underlying go-plugin handshake error (e.g. timeout, protocol mismatch) and fix per that root cause."],"exampleFix":"# before: 'failed to create executor: dial unix .../executor.sock: connect: connection refused'\n# after: reinstall/repair client binaries and restart\nsudo systemctl restart nomad\n# verify plugin loads\nnomad node status -verbose | grep -i driver","handlingStrategy":"try-catch","validationCode":"// preflight: confirm executor plugin exists and client is healthy\nif _, err := os.Stat(filepath.Join(nomadPluginDir, \"executor\")); err != nil {\n    return fmt.Errorf(\"executor plugin missing from plugin_dir: %w\", err)\n}\nif err := os.MkdirAll(taskDir, 0o755); err != nil {\n    return fmt.Errorf(\"cannot write executor.out in task dir: %w\", err)\n}","typeGuard":null,"tryCatchPattern":"_, _, err := driver.StartTask(cfg)\nif err != nil && strings.Contains(err.Error(), \"failed to create executor\") {\n    // restart/repair the Nomad client; check version skew\n    return RetryAfterClientRepair(fmt.Errorf(\"executor plugin failed to start: %w\", err))\n}","preventionTips":["Keep all Nomad binaries in a client at the same version.","Monitor client logs for go-plugin handshake errors.","Ensure plugin_dir contents are intact after upgrades.","Avoid launching Nomad clients in restricted containers that block subprocess spawning."],"tags":["nomad","java-driver","executor","go-plugin"],"backgroundTag":"executor-plugin-start-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}