{"record":{"id":"903e84d253387a0f","repo":"ory/hydra","slug":"newworker-failed-to-create-stdin-pipe","errorCode":null,"errorMessage":"newWorker: failed to create stdin pipe","messagePattern":"newWorker: failed to create stdin pipe","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"oryx/jsonnetsecure/jsonnet_pool.go","lineNumber":136,"sourceCode":"func newWorker(ctx context.Context) (_ worker, err error) {\n\ttracer := trace.SpanFromContext(ctx).TracerProvider().Tracer(\"\")\n\tctx, span := tracer.Start(ctx, \"jsonnetsecure.newWorker\")\n\tdefer otelx.End(span, &err)\n\n\tpath, _ := ctx.Value(contextValuePath).(string)\n\tif path == \"\" {\n\t\treturn worker{}, errors.New(\"newWorker: missing binary path in context\")\n\t}\n\targs, _ := ctx.Value(contextValueArgs).([]string)\n\tcmd := exec.Command(path, append(args, \"-0\")...)\n\tcmd.Env = []string{\"GOMAXPROCS=1\"}\n\tcmd.WaitDelay = 100 * time.Millisecond\n\n\tspan.SetAttributes(semconv.ProcessCommand(cmd.Path), semconv.ProcessCommandArgs(cmd.Args...))\n\n\tstdin, err := cmd.StdinPipe()\n\tif err != nil {\n\t\treturn worker{}, errors.Wrap(err, \"newWorker: failed to create stdin pipe\")\n\t}\n\n\tin := make(chan []byte, 1)\n\tgo func(c <-chan []byte) {\n\t\tfor input := range c {\n\t\t\tif _, err := stdin.Write(append(input, 0)); err != nil {\n\t\t\t\tstdin.Close()\n\t\t\t\treturn\n\t\t\t}\n\t\t}\n\t}(in)\n\n\tstdout, err := cmd.StdoutPipe()\n\tif err != nil {\n\t\treturn worker{}, errors.Wrap(err, \"newWorker: failed to create stdout pipe\")\n\t}\n\tstderr, err := cmd.StderrPipe()\n\tif err != nil {","sourceCodeStart":118,"sourceCodeEnd":154,"githubUrl":"https://github.com/ory/hydra/blob/4174065ffb052799890f7480f5360a877a67ffc1/oryx/jsonnetsecure/jsonnet_pool.go#L118-L154","documentation":"`newWorker` builds the child `exec.Cmd` for the jsonnet worker process; `cmd.StdinPipe()` failed, so no stdin pipe could be created and the worker construction aborts. In practice this almost always means the process is out of file descriptors, since pipe creation requires two fds.","triggerScenarios":"Called by the puddle pool constructor when spinning up a worker: `cmd.StdinPipe()` returns an error — overwhelmingly `EMFILE`/`ENFILE` (fd limit reached) from too many open files/sockets in the parent process.","commonSituations":"Large worker pool sizes combined with many open connections/files exhausting the fd limit; leaking worker processes/pipes over time so each new worker needs fds that no longer exist; low `ulimit -n` in containers.","solutions":["Check fd usage (`lsof -p <pid> | wc -l`) and raise the limit (`ulimit -n`, systemd LimitNOFILE, container ulimits)","Reduce the jsonnet worker pool size — NewProcessPool enforces a minimum of 5, so very small containers may need a bigger fd budget","Look for fd leaks: worker processes that are never destroyed keep pipes open; ensure the pool is Closed on shutdown","Restart the process to clear leaked fds if the pool has been running a long time"],"exampleFix":"// before: large pool on a box with low ulimit\npool := jsonnetsecure.NewProcessPool(100) // needs ~200+ fds\n// after: size the pool to the environment\npool := jsonnetsecure.NewProcessPool(runtime.NumCPU()) // and raise ulimit -n accordingly","handlingStrategy":"validation","validationCode":"// check fd headroom before creating the pool\nfds, _ := filepath.Glob(\"/proc/self/fd/*\")\nvar lim syscall.Rlimit\nsyscall.Getrlimit(syscall.RLIMIT_NOFILE, &lim)\nif len(fds) > int(lim.Cur)-2*poolSize {\n    return errors.New(\"not enough file descriptors for worker pool\")\n}","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Set a generous RLIMIT_NOFILE (e.g. 65536) for the service and containers","Call Pool.Close() on shutdown so worker pipes are released","Right-size the pool (each worker consumes several fds) instead of maxing it out","Alert on open-fd count approaching the limit in production"],"tags":["go","subprocess","file-descriptors","ulimit","pipe"],"backgroundTag":"too-many-open-files","analyzedSha":"4174065ffb052799890f7480f5360a877a67ffc1","analyzedAt":"2026-09-03T14:52:41.581Z","contentChangedAt":"2026-09-03T14:52:41.581Z","schemaVersion":2},"datasetVersion":"2026-09-10T17:17:09.494Z"}