{"record":{"id":"f647c51f3ea08dd0","repo":"nats-io/nats-server","slug":"unable-to-re-open-the-logfile-q-after-rotation","errorCode":null,"errorMessage":"Unable to re-open the logfile %q after rotation: %v","messagePattern":"Unable to re-open the logfile %q after rotation: (.+?)","errorType":"panic","errorClass":null,"httpStatus":null,"severity":"critical","filePath":"logger/log.go","lineNumber":269,"sourceCode":"\t\tl.out += int64(n)\n\t\tif l.out > l.limit {\n\t\t\tif err := l.f.Close(); err != nil {\n\t\t\t\tl.limit *= 2\n\t\t\t\tl.logDirect(l.l.errorLabel, \"Unable to close logfile for rotation (%v), will attempt next rotation at size %v\", err, l.limit)\n\t\t\t\tl.Unlock()\n\t\t\t\treturn n, err\n\t\t\t}\n\t\t\tfname := l.f.Name()\n\t\t\tnow := time.Now()\n\t\t\tbak := fmt.Sprintf(\"%s.%04d.%02d.%02d.%02d.%02d.%02d.%09d\", fname,\n\t\t\t\tnow.Year(), now.Month(), now.Day(), now.Hour(), now.Minute(),\n\t\t\t\tnow.Second(), now.Nanosecond())\n\t\t\tos.Rename(fname, bak)\n\t\t\tfileflags := os.O_WRONLY | os.O_APPEND | os.O_CREATE\n\t\t\tf, err := os.OpenFile(fname, fileflags, defaultLogPerms)\n\t\t\tif err != nil {\n\t\t\t\tl.Unlock()\n\t\t\t\tpanic(fmt.Sprintf(\"Unable to re-open the logfile %q after rotation: %v\", fname, err))\n\t\t\t}\n\t\t\tl.f = f\n\t\t\tn := l.logDirect(l.l.infoLabel, \"Rotated log, backup saved as %q\", bak)\n\t\t\tl.out = int64(n)\n\t\t\tl.limit = l.olimit\n\t\t\tif l.maxNumFiles > 0 {\n\t\t\t\tl.logPurge(fname)\n\t\t\t}\n\t\t}\n\t}\n\tl.Unlock()\n\treturn n, err\n}\n\nfunc (l *fileLogger) close() error {\n\tl.Lock()\n\tif l.closed {\n\t\tl.Unlock()","sourceCodeStart":251,"sourceCodeEnd":287,"githubUrl":"https://github.com/nats-io/nats-server/blob/3a66a489d262bf89b71a71c955c94920394532f3/logger/log.go#L251-L287","documentation":"When the log file hits its size limit, Logger.Write rotates it by renaming the current file and reopening the path with O_APPEND|O_CREATE. If the reopen fails (permissions, disk full, path vanished), the logger panics because it cannot continue logging safely, with this message carrying the underlying os error.","triggerScenarios":"Calling Write (any log emission) when rotation is pending and os.OpenFile(fname, O_WRONLY|O_APPEND|O_CREATE, defaultLogPerms) returns an error — e.g. the directory is read-only or was deleted, disk full, or the process lost permissions.","commonSituations":"Kubernetes container where the mounted log volume was remounted read-only; log directory permissions changed after rotation; disk quota exceeded on long-running servers.","solutions":["Fix filesystem permissions/ownership on the log file and its directory so the process can create files","Free disk space / resolve quota limits, then restart or let rotation retry","Point logfile to a writable volume and ensure the directory exists before starting the server"],"exampleFix":"// before (dir read-only)\nlogfile: \"/var/log/nats/nats.log\"\n// after (writable dir, correct perms)\nsudo chown nats:nats /var/log/nats\nlogfile: \"/var/log/nats/nats.log\"","handlingStrategy":"fallback","validationCode":"// pre-flight before starting with rotation\nf, err := os.OpenFile(logPath, os.O_WRONLY|os.O_APPEND|os.O_CREATE, 0o644)\nif err != nil { return fmt.Errorf(\"log path not writable: %v\", err) }\nf.Close()\ndf, err := checkDiskFree(filepath.Dir(logPath)) // ensure headroom > limit","typeGuard":null,"tryCatchPattern":"// Go: recover from rotation panic at a supervised boundary\ndev, err := server.NewLogger(...) // may panic on rotation\nif err != nil { ... }\n// run server under supervisor/systemd so a panic triggers restart,\n// and alert on \"Unable to re-open the logfile\" in crash output\ndefer func() {\n  if r := recover(); r != nil {\n    log.Printf(\"logger panicked: %v\", r) // then restart with file logging\n  }\n}()","preventionTips":["Ensure the log directory is writable by the service account and survives container restarts (persistent volume)","Monitor disk free space and set logfile size limits below available space","Run the server under a supervisor so a rotation panic restarts it with logging intact"],"tags":["logging","filesystem","panic","rotation"],"backgroundTag":"log-file-rotation-failure","analyzedSha":"3a66a489d262bf89b71a71c955c94920394532f3","analyzedAt":"2026-09-02T04:41:54.247Z","contentChangedAt":null,"schemaVersion":2},"datasetVersion":"2026-09-08T15:18:49.778Z"}