nats-io/nats-server · critical

Unable to re-open the logfile %q after rotation: %v

Error message

Unable to re-open the logfile %q after rotation: %v

What it means

When the log file hits its size limit, Logger.Write rotates it by renaming the current file and reopening the path with O_APPEND|O_CREATE. If the reopen fails (permissions, disk full, path vanished), the logger panics because it cannot continue logging safely, with this message carrying the underlying os error.

Source

Thrown at logger/log.go:269

		l.out += int64(n)
		if l.out > l.limit {
			if err := l.f.Close(); err != nil {
				l.limit *= 2
				l.logDirect(l.l.errorLabel, "Unable to close logfile for rotation (%v), will attempt next rotation at size %v", err, l.limit)
				l.Unlock()
				return n, err
			}
			fname := l.f.Name()
			now := time.Now()
			bak := fmt.Sprintf("%s.%04d.%02d.%02d.%02d.%02d.%02d.%09d", fname,
				now.Year(), now.Month(), now.Day(), now.Hour(), now.Minute(),
				now.Second(), now.Nanosecond())
			os.Rename(fname, bak)
			fileflags := os.O_WRONLY | os.O_APPEND | os.O_CREATE
			f, err := os.OpenFile(fname, fileflags, defaultLogPerms)
			if err != nil {
				l.Unlock()
				panic(fmt.Sprintf("Unable to re-open the logfile %q after rotation: %v", fname, err))
			}
			l.f = f
			n := l.logDirect(l.l.infoLabel, "Rotated log, backup saved as %q", bak)
			l.out = int64(n)
			l.limit = l.olimit
			if l.maxNumFiles > 0 {
				l.logPurge(fname)
			}
		}
	}
	l.Unlock()
	return n, err
}

func (l *fileLogger) close() error {
	l.Lock()
	if l.closed {
		l.Unlock()

View on GitHub (pinned to 3a66a489d2)

Solutions

1. Fix filesystem permissions/ownership on the log file and its directory so the process can create files
2. Free disk space / resolve quota limits, then restart or let rotation retry
3. Point logfile to a writable volume and ensure the directory exists before starting the server

Example fix

// before (dir read-only)
logfile: "/var/log/nats/nats.log"
// after (writable dir, correct perms)
sudo chown nats:nats /var/log/nats
logfile: "/var/log/nats/nats.log"

// pre-flight before starting with rotation
f, err := os.OpenFile(logPath, os.O_WRONLY|os.O_APPEND|os.O_CREATE, 0o644)
if err != nil { return fmt.Errorf("log path not writable: %v", err) }
f.Close()
df, err := checkDiskFree(filepath.Dir(logPath)) // ensure headroom > limit

// Go: recover from rotation panic at a supervised boundary
dev, err := server.NewLogger(...) // may panic on rotation
if err != nil { ... }
// run server under supervisor/systemd so a panic triggers restart,
// and alert on "Unable to re-open the logfile" in crash output
defer func() {
  if r := recover(); r != nil {
    log.Printf("logger panicked: %v", r) // then restart with file logging
  }
}()

When it happens

Trigger: Calling Write (any log emission) when rotation is pending and os.OpenFile(fname, O_WRONLY|O_APPEND|O_CREATE, defaultLogPerms) returns an error — e.g. the directory is read-only or was deleted, disk full, or the process lost permissions.

Common situations: Kubernetes container where the mounted log volume was remounted read-only; log directory permissions changed after rotation; disk quota exceeded on long-running servers.

Related errors

• can set log size limit only for file logger
• can set log max number of files only for file logger
• could not create storage directory - %v
• storage directory is not a directory
• storage directory is not writable

AI-assisted analysis of nats-io/nats-server@3a66a489d2 (2026-09-02). Data as JSON: /api/errors/f647c51f3ea08dd0. Report an issue: GitHub.