{"record":{"id":"6f371c0001f6c6ea","repo":"hashicorp/nomad","slug":"failed-to-create-fifo-for-extracting-logs-v","errorCode":null,"errorMessage":"failed to create fifo for extracting logs: %v","messagePattern":"failed to create fifo for extracting logs: (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"client/logmon/logmon.go","lineNumber":213,"sourceCode":"// processOutWriter to attach to the stdout or stderr of a process.\nfunc newLogRotatorWrapper(path string, logger hclog.Logger, rotator io.WriteCloser) (*logRotatorWrapper, error) {\n\tlogger.Debug(\"opening fifo\", \"path\", path)\n\n\tvar openFn func() (io.ReadCloser, error)\n\tvar err error\n\n\t_, serr := os.Stat(path)\n\tif os.IsNotExist(serr) {\n\t\topenFn, err = fifo.CreateAndRead(path)\n\t} else {\n\t\topenFn = func() (io.ReadCloser, error) {\n\t\t\treturn fifo.OpenReader(path)\n\t\t}\n\t}\n\n\tif err != nil {\n\t\tlogger.Error(\"failed to create FIFO\", \"stat_error\", serr, \"create_err\", err)\n\t\treturn nil, fmt.Errorf(\"failed to create fifo for extracting logs: %v\", err)\n\t}\n\n\twrap := &logRotatorWrapper{\n\t\tfifoPath:          path,\n\t\trotatorWriter:     rotator,\n\t\thasFinishedCopied: make(chan struct{}),\n\t\topenCompleted:     make(chan struct{}),\n\t\tlogger:            logger,\n\t}\n\n\twrap.start(openFn)\n\treturn wrap, nil\n}\n\n// start starts a goroutine that copies from the pipe into the rotator. This is\n// called by the constructor and not the user of the wrapper.\nfunc (l *logRotatorWrapper) start(openFn func() (io.ReadCloser, error)) {\n\tgo func() {","sourceCodeStart":195,"sourceCodeEnd":231,"githubUrl":"https://github.com/hashicorp/nomad/blob/482b49bf1aec006f089bcfc7e632d8f6ac303e5e/client/logmon/logmon.go#L195-L231","documentation":"newLogRotatorWrapper sets up the FIFO (named pipe) that task processes write logs into so the rotator can consume them; when the FIFO cannot be stat'd, created, or opened, this error is returned after logging the stat and create errors. Without the FIFO, the task's stdout/stderr cannot be captured.","triggerScenarios":"FIFO path invalid or in an unwritable directory (mkfifo fails), an incompatible non-FIFO file already exists at the path, os.Stat failing unexpectedly, or fifo.Open failing to open the created pipe.","commonSituations":"Stale FIFO file left by a previous crashed client with wrong permissions; log directory on a filesystem without FIFO support (some network/NFS mounts); resource exhaustion; client restarted into a dirty log dir.","solutions":["Inspect the logged stat_error and create_err fields for the underlying cause","Remove the stale/incompatible file at the FIFO path so mkfifo can recreate it","Ensure the log directory is writable and on a filesystem supporting named pipes (avoid exotic NFS configs)","Restart the client/logmon after cleaning the log directory so the FIFO is recreated","Check process privileges and SELinux/AppAllow policies for mkfifo in that path"],"exampleFix":"// before\nwrapper, err := newLogRotatorWrapper(\"/logs/task.fifo\", ...) // stale file blocks mkfifo\n// after\nif fi, err := os.Stat(\"/logs/task.fifo\"); err == nil && fi.Mode()&os.ModeNamedPipe == 0 {\n    os.Remove(\"/logs/task.fifo\") // clear incompatible leftover\n}\nwrapper, err := newLogRotatorWrapper(\"/logs/task.fifo\", ...)","handlingStrategy":"fallback","validationCode":"func ensureFifoCreatable(path string) error {\n    dir := filepath.Dir(path)\n    if err := os.MkdirAll(dir, 0o755); err != nil {\n        return fmt.Errorf(\"fifo dir not creatable: %w\", err)\n    }\n    if fi, err := os.Stat(path); err == nil && fi.Mode()&os.ModeNamedPipe == 0 {\n        if err := os.Remove(path); err != nil {\n            return fmt.Errorf(\"stale non-fifo file blocks mkfifo: %w\", err)\n        }\n    }\n    return nil\n}","typeGuard":null,"tryCatchPattern":"wrapper, err := newLogRotatorWrapper(fifoPath, logger, rotator)\nif err != nil {\n    if strings.Contains(err.Error(), \"failed to create fifo\") {\n        _ = os.Remove(fifoPath) // clear stale path and retry once\n        wrapper, err = newLogRotatorWrapper(fifoPath, logger, rotator)\n    }\n    if err != nil {\n        return nil, err\n    }\n}","preventionTips":["Clean stale FIFOs from crashed clients on startup","Put log/FIFO paths on local filesystems that support named pipes","Ensure the logmon user has mkfifo rights (SELinux/AppArmor)","Log and inspect stat_error/create_err pairs when diagnosing"],"tags":["logging","fifo","logmon","filesystem"],"backgroundTag":"fifo-creation-failed","analyzedSha":"482b49bf1aec006f089bcfc7e632d8f6ac303e5e","analyzedAt":"2026-09-04T07:54:14.808Z","contentChangedAt":"2026-09-04T07:54:14.808Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}