argoproj/argo-workflows · error

supervisor reported pre-main failure: %s

Error message

supervisor reported pre-main failure: %s

What it means

The init-less supervisor writes a status marker whose first line is a token (RUNNING, READY, FAILED). If the token is FAILED, the supervisor's pre-main phase (e.g. artifact loading, template setup) failed, and the emissary surfaces the supervisor's own message via this error, failing the step before main ever runs.

Source

Thrown at cmd/argoexec/commands/emissary.go:542

			return false, nil
		}
		return true, fmt.Errorf("stat supervisor status: %w", statErr)
	}
	body, readErr := os.ReadFile(statusPath)
	if readErr != nil {
		// Stat just succeeded, so a read failure here means we raced the
		// supervisor's atomic rename (the old inode vanished between stat and
		// read). Treat it as transient and re-evaluate on the next tick/event
		// rather than failing the wait.
		//nolint:nilerr // deliberate: swallow the transient read error and retry
		return false, nil
	}
	token, message := parseSupervisorStatus(body)
	switch token {
	case statusReady:
		return true, nil
	case statusFailed:
		return true, fmt.Errorf("supervisor reported pre-main failure: %s", message)
	default:
		// RUNNING, or a transient/partial read: the supervisor is alive only if
		// it is still heartbeating, i.e. the marker's mtime is fresh.
		if time.Since(fi.ModTime()) > timeout {
			return true, fmt.Errorf("supervisor presumed dead: no status update within %s", timeout)
		}
		return false, nil
	}
}

// parseSupervisorStatus splits the marker into its first-line token and the
// remaining message (used by the FAILED token to carry the cause).
func parseSupervisorStatus(body []byte) (token, message string) {
	first, rest, _ := strings.Cut(string(body), "\n")
	return strings.TrimSpace(first), strings.TrimSpace(rest)
}

// waitForDependencyExitCode blocks until the given dependency exitcode file is

View on GitHub (pinned to 35bff19146)

Solutions

1. Read the message after the colon in the full error — it is the supervisor's own failure reason and points at the actual cause
2. Verify artifact source credentials/paths (s3/gcs/artifactory config in the workflow's artifactRepository or artifact locators)
3. Check pod network policy allows egress to the artifact storage endpoint
4. Inspect argoexec and supervisor logs for the underlying stack before the FAILED marker was written

# Pre-submit checks that pre-main cannot fail:
argo lint my-workflow.yaml
# verify artifact source reachability and credentials:
argo archive list  # or test the artifact repository creds out-of-band
kubectl auth can-i get secrets  # if artifacts pull credentials from secrets

if err := waitForSupervisorReady(ctx); err != nil {
    var supErr *SupervisorError
    if errors.As(err, &supErr) && strings.HasPrefix(supErr.Message, "supervisor reported pre-main failure:") {
        log.Printf("pre-main failed: %s — inspect artifact sources and template", supErr.Message)
    }
    return err
}

When it happens

Trigger: The status marker contains FAILED, with the supervisor's failure reason as the message — typically a failed input-artifact download, template unmarshal error, or other pre-main setup failure in the supervisor.

Common situations: Input artifact source unavailable (bad S3/GCS credentials, deleted artifact); malformed template passed via /var/run/argo/template; network egress blocked from the pod to artifact storage.

Related errors

• watching supervisor status: %w
• supervisor presumed dead: status marker never appeared withi
• stat supervisor status: %w
• supervisor presumed dead: no status update within %s
• failed to re-enter working directory %q after staging input

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/9c45e7e73c4e772c. Report an issue: GitHub.