{"record":{"id":"4cae13305a22048a","repo":"hashicorp/nomad","slug":"getter-subprocess-failed-v-v","errorCode":null,"errorMessage":"getter subprocess failed: %v: %v","messagePattern":"getter subprocess failed: (.+?): (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/allocrunner/taskrunner/getter/util.go","lineNumber":274,"sourceCode":"\t// final method of ensuring subprocess termination\n\tctx, cancel := subproc.Context(env.deadline())\n\tdefer cancel()\n\n\t// start the subprocess, passing in parameters via stdin\n\toutput := new(bytes.Buffer)\n\tcmd := exec.CommandContext(ctx, bin, SubCommand)\n\tcmd.Env = environment(env.TaskDir, env.SetEnvironmentVariables)\n\tcmd.Stdin = env.reader()\n\tcmd.Stdout = output\n\tcmd.Stderr = output\n\n\t// start & wait for the subprocess to terminate\n\tif err := cmd.Run(); err != nil {\n\t\tmsg := subproc.Log(output, s.logger.Error)\n\n\t\treturn &Error{\n\t\t\tURL:         env.Source,\n\t\t\tErr:         fmt.Errorf(\"getter subprocess failed: %v: %v\", err, msg),\n\t\t\tRecoverable: true,\n\t\t}\n\t}\n\tsubproc.Log(output, s.logger.Debug)\n\n\t// if no root has been defined, no inspection\n\t// is being performed so return now.\n\tif at == nil {\n\t\treturn nil\n\t}\n\n\t// generate the inspector for the destination\n\tartifactInspector, err := genWalkInspector(env.Destination)\n\tif err != nil {\n\t\treturn err\n\t}\n\n\t// inspect the contents to find any unwanted files","sourceCodeStart":256,"sourceCodeEnd":292,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/allocrunner/taskrunner/getter/util.go#L256-L292","documentation":"runCmd runs the go-getter as a subprocess with a per-artifact environment; if cmd.Run() returns non-zero, Nomad collects the subprocess log output and wraps both the error and the log message in an Error with Recoverable: true, meaning the task runner may retry the artifact fetch. The message intentionally pairs the exit error with the getter's own log output because go-getter failures (auth, DNS, TLS, checksum) surface only in the subprocess output.","triggerScenarios":"The getter subprocess exits non-zero — unreachable host/DNS failure, HTTP 403/404, bad S3/GCS credentials, TLS certificate errors, unsupported URL scheme, checksum mismatch, or the source binary (git/ssh) missing on the client host","commonSituations":"Private repos without deploy keys on the Nomad client, expired cloud credentials, artifacts behind a proxy not configured on the node, S3 region/bucket typos, or a client missing git/curl-based getter dependencies","solutions":["Read the appended msg (subprocess log) in the error to find the root cause — it names the underlying getter failure","Verify the source URL is reachable from the Nomad client node (curl the URL as the nomad user)","Fix credentials: for git sources ensure SSH keys are in the client's ssh-agent or GIT_SSH_COMMAND is set; for s3/gcs verify provider config","Since Recoverable is true, simply retrying after fixing network/auth will usually resolve it; check client connectivity/DNS if transient"],"exampleFix":"// before (client without git access to private repo)\nsource = \"git::ssh://git@private.example.com/repo.git\"\n// after (configure key on client, e.g. via client config or env)\n// ssh-keyscan private.example.com >> ~nomad/.ssh/known_hosts\n// load deploy key into nomad user's ssh-agent, then keep the same source","handlingStrategy":"retry","validationCode":"// pre-flight: check URL reachability from the client node\ncurl -sSIf -o /dev/null --max-time 10 https://example.com/artifact.tgz","typeGuard":null,"tryCatchPattern":"// Recoverable=true: retry with backoff, log subprocess output\nif e, ok := err.(*getter.Error); ok && e.Recoverable {\n    backoff(e, e.Err) // includes getter subprocess log msg\n}","preventionTips":["Curl the artifact URL as the nomad user on the client before scheduling","Keep git/ssh deploy keys and known_hosts configured on clients","Rotate cloud credentials before expiry; test s3:// gs:// sources","Ensure client has network egress/proxy settings for artifact hosts"],"tags":["artifacts","subprocess","network","retryable"],"backgroundTag":"artifact-download-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}