argoproj/argo-workflows · critical

failed to read template: %w

Error message

failed to read template: %w

What it means

runEmissary needs the Workflow Template (the pod's template JSON) to know ContainerSet dependencies and artifact staging paths. readTemplate() first tries the file /var/run/argo/template (written by the init container, or by the supervisor in init-less mode) and falls back to the ARGO_TEMPLATE env var. This error is thrown when neither source is readable — the file read failed with an error other than NotExist (e.g. permission denied), or the file is missing AND ARGO_TEMPLATE is not set/resolvable.

Source

Thrown at cmd/argoexec/commands/emissary.go:155

	// the template. Gated on an env var so legacy pods are unaffected.
	waitForReady := os.Getenv(common.EnvVarWaitForReady) == "true"
	if waitForReady {
		if waitErr := waitForSupervisorReady(ctx); waitErr != nil {
			// Distinct exit code so the controller attributes the failure
			// to supervisor pre-main setup rather than the user command.
			// The process exit code (not just the exitcode file) must carry
			// the sentinel, because inferFailedReason keys off the container's
			// terminated exit code; wrap so main propagates 65 while keeping
			// waitErr's message.
			exitCode = common.ExitCodeSupervisorPreMainFailure
			logger.WithError(waitErr).Error(ctx, "supervisor failed before main container started")
			return argoerrors.NewExitErrWithCause(exitCode, waitErr)
		}
	}

	data, err := readTemplate()
	if err != nil {
		return fmt.Errorf("failed to read template: %w", err)
	}

	template := &wfv1.Template{}
	if err = json.Unmarshal(data, template); err != nil {
		return fmt.Errorf("failed to unmarshal template: %w", err)
	}

	// In init-less pod mode, main can't use the legacy per-artifact
	// SubPath bind mount (kubelet races the supervisor's write). The
	// input-artifacts volume is mounted whole at /argo/inputs/artifacts
	// and the emissary symlinks each input artifact into its expected
	// path once supervisor has finished writing (guaranteed by the
	// ready-marker wait above). Only `main` runs this — ContainerSet
	// children and sidecars don't get artifact paths symlinked in.
	if waitForReady && containerName == common.MainContainerName {
		if stageErr := stageInputArtifacts(ctx, template); stageErr != nil {
			// As above: propagate the sentinel as the process exit code so
			// inferFailedReason attributes this to supervisor pre-main setup.

View on GitHub (pinned to 35bff19146)

Solutions

1. Check the pod spec: the main/supervisor container must have ARGO_TEMPLATE set or the /var/run/argo volume mounted where the template file is written.
2. Verify supervisor/init completed: `kubectl exec <pod> -c init/supervisor -- ls -l /var/run/argo/template` and check its logs.
3. Align controller and executor images to the same Argo Workflows version.
4. Ensure waitForReady/supervisor readiness gating is intact (emissary waits on the ready marker before reading) — do not patch it out.
5. Confirm the config-map offload dir (common.EnvConfigMountPath) is mounted if ARGO_TEMPLATE uses the offload sentinel.

Example fix

// before: custom patch removed the template env from main
containers:
  - name: main
    env: []            # ARGO_TEMPLATE gone, /var/run/argo/template not written yet
// after: leave controller-managed env/volumes intact
containers:
  - name: main
    env:
      - name: ARGO_TEMPLATE
        valueFrom: { ... }   # controller-injected

// Detect a usable template source before proceeding:
func templateSourceAvailable() error {
	if _, err := os.Stat("/var/run/argo/template"); err == nil {
		return nil
	}
	if _, ok := os.LookupEnv("ARGO_TEMPLATE"); ok {
		return nil
	}
	return fmt.Errorf("neither /var/run/argo/template nor ARGO_TEMPLATE available")
}

data, err := readTemplate()
if err != nil {
	if errors.Is(err, os.ErrPermission) {
		logger.WithError(err).Error(ctx, "template file unreadable — check /var/run/argo mount and securityContext")
	} else if strings.Contains(err.Error(), "neither") {
		logger.WithError(err).Error(ctx, "no template source: init/supervisor did not write it and ARGO_TEMPLATE unset — check pod spec and version skew")
	}
	return err
}

When it happens

Trigger: ARGO_template file missing and ARGO_TEMPLATE unset (pod created by a controller path that expected the file only); /var/run/argo/template unreadable due to permissions; in init-less mode the supervisor never wrote the template before emissary read it and the ARGO_TEMPLATE fallback is absent; the template-env offload sentinel cannot be resolved from the config mount.

Common situations: Version skew between controller and executor around init-less/supervisor mode; custom pod patches removing the /var/run/argo volume or the ARGO_TEMPLATE env; pod started before init container completed in a modified layout; securityContext blocking /var/run/argo reads.

Related errors

• failed to read container args file %s: %w
• failed to unmarshal container args: %w
• failed to write large arg %d to file: %w
• failed to start command: %w
• neither %s nor %s is available

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/3365bc4d6fab565c. Report an issue: GitHub.