hashicorp/nomad · error

failed to create stdout logfile for %q: %v

Error message

failed to create stdout logfile for %q: %v

What it means

logmon.NewTaskLogger creates the stdout file rotator for a task before wiring the FIFO wrapper; if logging.NewFileRotator fails (cannot create/open the stdout log file), the task logger creation aborts with this wrapped error naming the stdout log file. It means log files could not be set up in the task's log directory.

Source

Thrown at client/logmon/logmon.go:144

			tl.lro.Close()
		})
	}
	if tl.lre != nil {
		wg.Go(func() {
			tl.lre.Close()
		})
	}
	wg.Wait()
}

func NewTaskLogger(cfg *LogConfig, logger hclog.Logger) (*TaskLogger, error) {
	tl := &TaskLogger{config: cfg}

	logFileSize := int64(cfg.MaxFileSizeMB * 1024 * 1024)
	lro, err := logging.NewFileRotator(cfg.LogDir, cfg.StdoutLogFile,
		cfg.MaxFiles, logFileSize, logger)
	if err != nil {
		return nil, fmt.Errorf("failed to create stdout logfile for %q: %v", cfg.StdoutLogFile, err)
	}

	wrapperOut, err := newLogRotatorWrapper(cfg.StdoutFifo, logger, lro)
	if err != nil {
		return nil, err
	}

	tl.lro = wrapperOut

	lre, err := logging.NewFileRotator(cfg.LogDir, cfg.StderrLogFile,
		cfg.MaxFiles, logFileSize, logger)
	if err != nil {
		return nil, fmt.Errorf("failed to create stderr logfile for %q: %v", cfg.StderrLogFile, err)
	}

	wrapperErr, err := newLogRotatorWrapper(cfg.StderrFifo, logger, lre)
	if err != nil {
		return nil, err

View on GitHub (pinned to 482b49bf1a)

Solutions

1. Verify cfg.LogDir exists, is writable by the client process, and has free space
2. Check the wrapped error: EACCES -> fix ownership/permissions, ENOSPC -> free disk, ENOENT -> create/mount the dir
3. Correct client config (log_dir, max_files, max_file_size) in the agent configuration
4. Check SELinux/AppArmor policies permitting writes to the log directory
5. Run logmon with the same effective user that owns the task log dir

Example fix

// before
lro, err := logging.NewFileRotator("/nonexistent/dir", name, ...) // ENOENT
// after
if err := os.MkdirAll(cfg.LogDir, 0o755); err != nil {
    return nil, fmt.Errorf("cannot create log dir: %w", err)
}
lro, err := logging.NewFileRotator(cfg.LogDir, name, ...)

func ensureLogDirUsable(dir string) error {
    if err := os.MkdirAll(dir, 0o755); err != nil {
        return fmt.Errorf("log dir not creatable: %w", err)
    }
    probe := filepath.Join(dir, ".probe")
    if err := os.WriteFile(probe, []byte("x"), 0o600); err != nil {
        return fmt.Errorf("log dir not writable: %w", err)
    }
    return os.Remove(probe)
}

tl, err := NewTaskLogger(cfg)
if err != nil {
    if strings.Contains(err.Error(), "failed to create stdout logfile") {
        return fmt.Errorf("check log_dir permissions and disk space: %w", err)
    }
    return err
}

When it happens

Trigger: NewFileRotator failing for the stdout file: LogDir does not exist or is not writable, disk full, path too long, or permission denied on the log directory; invalid MaxFiles/MaxFileSizeMB configuration.

Common situations: Log directory volume not mounted or owned by another user in containers; read-only filesystem; disk quota exceeded; misconfigured client "log_dir" option; SELinux blocking writes to the log path.

Related errors

• failed to create stderr logfile for %q: %v
• failed to create fifo for extracting logs: %v
• No nomad log file defined
• No nomad log file defined
• error creating fifo %v: %w

AI-assisted analysis of hashicorp/nomad@482b49bf1a (2026-09-04). Data as JSON: /api/errors/daad630f937ba7c8. Report an issue: GitHub.