kubernetes/kops · error

unable to parse service ExecMainStartTimestamp %q: %v

Error message

unable to parse service ExecMainStartTimestamp %q: %v

What it means

The SmartRestart feature compares each dependency service's start time against the newest dependency change. It parses systemd's ExecMainStartTimestamp using the layout "Mon 2006-01-02 15:04:05 MST"; if systemd emits a timestamp in a different format/locale, time.Parse fails and the task aborts rather than making an unsafe restart decision.

Source

Thrown at upup/pkg/fi/nodeup/nodetasks/service.go:380

				modTime := stat.ModTime()
				if newest.IsZero() || newest.Before(modTime) {
					newest = modTime
				}
			}

			if !newest.IsZero() {
				properties, err := getSystemdStatus(e.Name)
				if err != nil {
					return err
				}

				startedAt := properties["ExecMainStartTimestamp"]
				if startedAt == "" {
					klog.Warningf("service was running, but did not have ExecMainStartTimestamp: %q", serviceName)
				} else {
					startedAtTime, err := time.Parse("Mon 2006-01-02 15:04:05 MST", startedAt)
					if err != nil {
						return fmt.Errorf("unable to parse service ExecMainStartTimestamp %q: %v", startedAt, err)
					}
					if startedAtTime.Before(newest) {
						klog.V(2).Infof("will restart service %q because dependency changed after service start", serviceName)
						action = "restart"
					} else {
						klog.V(2).Infof("will not restart service %q - started after dependencies", serviceName)
					}
				}
			}
		}
	}

	if action != "" && fi.ValueOf(e.ManageState) {
		args := []string{"systemctl", action, serviceName}
		// We use --no-block to avoid hanging if the service has issues stopping/starting
		args = append(args, "--no-block")
		cmd := exec.Command(args[0], args[1:]...)
		klog.Infof("Restarting service %q (running %q)", serviceName, strings.Join(cmd.Args, " "))

View on GitHub (pinned to 4c8573c808)

Solutions

1. Force a C/POSIX locale for the systemctl invocation or the node (e.g. LC_ALL=C systemctl show ...)
2. Upgrade or patch nodeup to parse the timestamp with a more tolerant layout or use ExecMainStartTimestampMonotonic/usec instead
3. Disable SmartRestart (set it to false) on the Service task to skip timestamp parsing
4. Check systemd version on the node; align nodeup with the distro's systemd output format

Example fix

// before
startedAtTime, err := time.Parse("Mon 2006-01-02 15:04:05 MST", startedAt)
// after
usec := properties["ExecMainStartTimestampMonotonic"] // or parse numeric timestamp
startedAtTime, err := parseSystemdTimestamp(startedAt) // tolerant multi-layout parser

// Detect risky locale on the node beforehand:
[ "$LC_ALL$LC_TIME" = '' ] || echo 'non-C locale may break ExecMainStartTimestamp parsing'

if err := task.RenderLocal(...); err != nil {
	if strings.Contains(err.Error(), "ExecMainStartTimestamp") {
		// fall back: disable SmartRestart or restart unconditionally
		klog.Warningf("smart restart unavailable: %v", err)
		return restartService(serviceName) // fallback action
	}
	return err
}

When it happens

Trigger: After `systemctl show <dep>`, ExecMainStartTimestamp is non-empty but does not match the hardcoded layout — e.g. non-English locale (localized month/timezone names), unusual timezone abbreviations like "-03" numeric offsets, or format differences across systemd versions.

Common situations: Nodes with LC_TIME set to a non-C locale, or systemd versions/patches that render timestamps differently than the assumed US-English format, breaking SmartRestart on dependency changes.

Understand the failure class

• Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

• error building kubelet flags: %v
• failed to create bpf mount unit: %w
• failed to create cgroupv2 mount unit: %w
• error running tasks: %w
• error doing systemd show %s: %v Output: %s

AI-assisted analysis of kubernetes/kops@4c8573c808 (2026-09-05). Data as JSON: /api/errors/57fbef7b4b0e3dba. Report an issue: GitHub.