{"record":{"id":"d59336f0f5821ac9","repo":"ory/hydra","slug":"newworker-failed-to-create-stdout-pipe","errorCode":null,"errorMessage":"newWorker: failed to create stdout pipe","messagePattern":"newWorker: failed to create stdout pipe","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"oryx/jsonnetsecure/jsonnet_pool.go","lineNumber":151,"sourceCode":"\n\tstdin, err := cmd.StdinPipe()\n\tif err != nil {\n\t\treturn worker{}, errors.Wrap(err, \"newWorker: failed to create stdin pipe\")\n\t}\n\n\tin := make(chan []byte, 1)\n\tgo func(c <-chan []byte) {\n\t\tfor input := range c {\n\t\t\tif _, err := stdin.Write(append(input, 0)); err != nil {\n\t\t\t\tstdin.Close()\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t}(in)\n\n\tstdout, err := cmd.StdoutPipe()\n\tif err != nil {\n\t\treturn worker{}, errors.Wrap(err, \"newWorker: failed to create stdout pipe\")\n\t}\n\tstderr, err := cmd.StderrPipe()\n\tif err != nil {\n\t\treturn worker{}, errors.Wrap(err, \"newWorker: failed to create stderr pipe\")\n\t}\n\n\tif err := cmd.Start(); err != nil {\n\t\treturn worker{}, errors.Wrap(err, \"newWorker: failed to start process\")\n\t}\n\n\tspan.SetAttributes(semconv.ProcessPID(cmd.Process.Pid))\n\n\tscan := func(c chan<- string, r io.Reader, maxTokenSize int) {\n\t\tdefer close(c)\n\t\tscanner := bufio.NewScanner(r)\n\t\tscanner.Buffer(make([]byte, 0, 64*KiB), maxTokenSize)\n\n\t\tscanner.Split(splitNull)","sourceCodeStart":133,"sourceCodeEnd":169,"githubUrl":"https://github.com/ory/hydra/blob/4174065ffb052799890f7480f5360a877a67ffc1/oryx/jsonnetsecure/jsonnet_pool.go#L133-L169","documentation":"`newWorker` calls `cmd.StdoutPipe()` to receive evaluation results from the jsonnet child; when that returns an error the worker cannot be built and this wrapped error is returned. Like the stdin pipe failure, this is essentially always fd exhaustion (a pipe requires two file descriptors).","triggerScenarios":"During pool warm-up or on-demand worker creation via puddle's Constructor: `cmd.StdoutPipe()` returns an error, typically EMFILE/ENFILE because the parent has exhausted its file descriptors.","commonSituations":"Running in a container with a low nofile limit while holding many DB connections, sockets, and worker pipes; a leak of pipes from workers that crashed without destroy; spiking the pool size under load.","solutions":["Raise the fd limit (`ulimit -n`, systemd LimitNOFILE=65536, Docker/K8s `ulimits`/securityContext)","Check for pipe/fd leaks with `ls /proc/<pid>/fd | wc -l` and close the pool properly on shutdown","Lower the worker pool size to fit the fd budget","Audit other subsystems (HTTP clients, DB pools) for unclosed connections competing for fds"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"var lim syscall.Rlimit\nsyscall.Getrlimit(syscall.RLIMIT_NOFILE, &lim)\nopenFDs := countOpenFDs() // e.g. len of /proc/self/fd entries\nif openFDs+2 >= int(lim.Cur) {\n    return errors.New(\"fd limit nearly exhausted; cannot create stdout pipe\")\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Raise ulimit -n / systemd LimitNOFILE in every environment (CI, staging, prod)","Fix fd leaks: ensure pool Close and worker destroy are invoked","Reduce concurrent subprocess pools; reuse one shared pool","Monitor /proc/<pid>/fd count with alerts at ~80% of the limit"],"tags":["go","subprocess","file-descriptors","ulimit","pipe"],"backgroundTag":"too-many-open-files","analyzedSha":"4174065ffb052799890f7480f5360a877a67ffc1","analyzedAt":"2026-09-03T14:52:41.581Z","contentChangedAt":"2026-09-03T14:52:41.581Z","schemaVersion":2},"datasetVersion":"2026-09-10T17:17:09.494Z"}