failed to read working directory before staging input artifa

Error message

failed to read working directory before staging input artifacts: %w

What it means

Before staging (symlinking) input artifacts, the executor records the current working directory via os.Getwd so it can return there afterwards. This error means Getwd itself failed — the process's working directory no longer exists (getcwd() fails on a deleted cwd) or traversal permissions changed. The executor deliberately fails fast here because continuing would leave relative paths unresolvable.

Source

Thrown at cmd/argoexec/commands/emissary.go:350

	return common.ResolveTemplateEnvValue(envVal, offloadDir)
}

func stageInputArtifacts(ctx context.Context, tmpl *wfv1.Template) error {
	return stageInputArtifactsAt(ctx, common.ExecutorArtifactBaseDir, tmpl)
}

// stageInputArtifactsAt is the parameterized form used by tests; production
// calls stageInputArtifacts with the constants. It links each input artifact
// into place and re-enters the working directory afterwards, stepping off it
// for the duration: linking replaces the cwd when an artifact's path is the
// container's workingDir, Windows refuses to delete a directory in use as a
// working directory, and a child forked with the deleted directory as cwd
// would see getcwd() fail and relative paths resolve to nothing. The final
// chdir follows the symlink to whatever now sits at the path.
func stageInputArtifactsAt(ctx context.Context, baseDir string, tmpl *wfv1.Template) error {
	origWd, err := os.Getwd()
	if err != nil {
		return fmt.Errorf("failed to read working directory before staging input artifacts: %w", err)
	}
	if err := os.Chdir(varRunArgo); err != nil {
		return fmt.Errorf("failed to leave working directory before staging input artifacts: %w", err)
	}
	if err := linkInputArtifactsAt(ctx, baseDir, tmpl); err != nil {
		return err
	}
	if err := os.Chdir(origWd); err != nil {
		return fmt.Errorf("failed to re-enter working directory %q after staging input artifacts (an input artifact staged at the workingDir path must be a directory): %w", origWd, err)
	}
	return nil
}

// linkInputArtifactsAt creates a symlink at each input artifact's path
// pointing to the file that supervisor wrote under /argo/inputs/artifacts/
// <name>. This replaces the legacy SubPath bind-mount-per-artifact scheme,
// which can't be used in init-less mode because kubelet pre-creates SubPath
// entries as empty directories before supervisor can write the real file.

View on GitHub (pinned to 35bff19146)

Solutions

Ensure nothing deletes or recreates the template's workingDir directory while the pod runs.
Set an explicit, existing workingDir in the template instead of relying on the image default.
If a previous step must clear the directory, delete contents (rm -rf dir/*) rather than the directory itself.
Check the wrapped inner error (ENOENT vs EACCES) to distinguish deleted cwd from permission problems.
Restart the workflow pod so argoexec starts with a valid cwd.

Example fix

// before
echo done > /dev/null && rm -rf /work   # deletes the pod's cwd
// after
rm -rf /work/*   # clear contents, keep the directory

Defensive patterns

Strategy: validation

Validate before calling

// Ensure the working directory exists before staging:
stat "$PWD" >/dev/null 2>&1 || { echo "cwd missing" >&2; exit 1; }

Try / catch

try {
  await stageArtifacts(tmpl)
} catch (e) {
  if (String(e).includes('failed to read working directory')) {
    // recreate workingDir and restart the pod
  }
}

Prevention

Never rm -rf the workingDir itself; delete only its contents.
Set an explicit workingDir that exists in the image.
Avoid steps that recreate mounted directories while the pod is running.

When it happens

Trigger: os.Getwd() returns an error inside stageInputArtifactsAt — usually because a parent container process deleted/replaced the working directory (e.g. a previous step rm -rf'ed the mounted dir) before the emissary process staged artifacts, or PWD env is stale and the path cannot be resolved.

Common situations: Templates whose workingDir points at a volume mount that a preStep or earlier retry wiped; init-container ordering where the workdir is recreated after argoexec started; k8s subPath mounts deleted and recreated out from under the running pod.

Related errors

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/69310fb0d6e5d3bd. Report an issue: GitHub.