argoproj/argo-workflows · error

mkdir %s error: %w

Error message

mkdir %s error: %w

What it means

During OSS Load, after fetching the object fails, the driver ensures the destination directory exists via os.MkdirAll(filepath.Dir(path), 0o700) before a possible directory download. If creating that local directory fails (permissions, read-only FS, path is a file), Load retries then surfaces 'mkdir <dir> error'.

Source

Thrown at workflow/artifacts/oss/oss.go:149

			logging.RequireLoggerFromContext(ctx).WithFields(logging.Fields{"path": path, "key": inputArtifact.OSS.Key}).Info(ctx, "OSS Load")
			osscli, err := ossDriver.newOSSClient(ctx)
			if err != nil {
				return !isTransientOSSErr(ctx, err), err
			}
			bucketName := inputArtifact.OSS.Bucket
			err = setBucketLogging(osscli, bucketName)
			if err != nil {
				return !isTransientOSSErr(ctx, err), err
			}
			bucket, err := osscli.Bucket(bucketName)
			if err != nil {
				return !isTransientOSSErr(ctx, err), err
			}
			objectName := inputArtifact.OSS.Key
			dirPath := filepath.Dir(path)
			err = os.MkdirAll(dirPath, 0o700)
			if err != nil {
				return false, fmt.Errorf("mkdir %s error: %w", dirPath, err)
			}
			origErr := bucket.GetObjectToFile(objectName, path)
			if origErr == nil {
				return true, nil
			}
			if !IsOssErrCode(origErr, "NoSuchKey") {
				return !isTransientOSSErr(ctx, origErr), fmt.Errorf("failed to get file: %w", origErr)
			}
			// If we get here, the error was a NoSuchKey. The key might be a oss "directory"
			isDir, err := IsOssDirectory(bucket, objectName)
			if err != nil {
				return !isTransientOSSErr(ctx, err), fmt.Errorf("failed to test if %s/%s is a directory: %w", bucketName, objectName, err)
			}
			if !isDir {
				// It's neither a file, nor a directory. Return the original NoSuchKey error
				return false, origErr
			}

View on GitHub (pinned to 35bff19146)

Solutions

1. Point the input artifact `path` at a writable location, e.g. /tmp/... or a writable emptyDir/PVC mount.
2. Remove readOnly: true from the volumeMount backing the artifact path, or mount a second writable volume.
3. Ensure the container's runAsUser/runAsGroup has write permission on the destination parent.
4. Check that the destination parent is not an existing file (delete or rename the conflicting path).

Example fix

// before
inputs:
  artifacts:
    - name: data
      path: /mnt/data/read-only/input.json  # readonly mount
// after
inputs:
  artifacts:
    - name: data
      path: /work/input.json   # writable volume

func validateWritableDest(path string) error {
	dir := filepath.Dir(path)
	if fi, err := os.Stat(dir); err == nil && !fi.IsDir() {
		return fmt.Errorf("%s exists and is not a directory", dir)
	}
	probe := filepath.Join(dir, ".argo-write-probe")
	if err := os.WriteFile(probe, nil, 0o600); err != nil {
		return fmt.Errorf("destination %s not writable: %w", dir, err)
	}
	_ = os.Remove(probe)
	return nil
}

err := driver.Load(ctx, art, path)
if err != nil && strings.Contains(err.Error(), "mkdir ") {
	// destination parent unwritable: remount writable volume or change path
}

When it happens

Trigger: Downloading an OSS artifact to a local path whose parent directory cannot be created: the output mount is read-only, the pod user lacks write permission on the parent (e.g. writing to / or /etc), the parent path exists as a regular file, or disk is full.

Common situations: Empty-volume mount paths in containers where the artifact path resolves to a system directory; running the executor as non-root while the artifact path defaults to a root-owned location; a volumeMount with readOnly: true on the destination path.

Related errors

• unable to create local directory %s: %w
• unable to open file %s: %w
• failed to test if %s/%s is a directory: %w
• failed get directory: %w
• failed to create folder path: %w

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/997ec739023a09a1. Report an issue: GitHub.