argoproj/argo-workflows · error

failed get directory: %w

Error message

failed get directory: %w

What it means

When the missing OSS key turns out to be a directory prefix, Load downloads the whole prefix via GetOssDirectory (recursively listing and fetching every object under it). Any failure inside that recursive download — a nested object 403/404, listing pagination failure, or local write error — is wrapped as 'failed get directory'.

Source

Thrown at workflow/artifacts/oss/oss.go:169

			origErr := bucket.GetObjectToFile(objectName, path)
			if origErr == nil {
				return true, nil
			}
			if !IsOssErrCode(origErr, "NoSuchKey") {
				return !isTransientOSSErr(ctx, origErr), fmt.Errorf("failed to get file: %w", origErr)
			}
			// If we get here, the error was a NoSuchKey. The key might be a oss "directory"
			isDir, err := IsOssDirectory(bucket, objectName)
			if err != nil {
				return !isTransientOSSErr(ctx, err), fmt.Errorf("failed to test if %s/%s is a directory: %w", bucketName, objectName, err)
			}
			if !isDir {
				// It's neither a file, nor a directory. Return the original NoSuchKey error
				return false, origErr
			}

			if err = GetOssDirectory(ctx, bucket, objectName, path); err != nil {
				return !isTransientOSSErr(ctx, err), fmt.Errorf("failed get directory: %w", err)
			}
			return true, nil
		})
	return err
}

// OpenStream opens a stream reader for an artifact from OSS compliant storage
func (ossDriver *ArtifactDriver) OpenStream(ctx context.Context, inputArtifact *wfv1.Artifact) (io.ReadCloser, error) {
	var stream io.ReadCloser
	err := waitutil.Backoff(defaultRetry,
		func() (bool, error) {
			logging.RequireLoggerFromContext(ctx).WithField("key", inputArtifact.OSS.Key).Info(ctx, "OSS OpenStream")
			osscli, err := ossDriver.newOSSClient(ctx)
			if err != nil {
				return !isTransientOSSErr(ctx, err), err
			}
			bucketName := inputArtifact.OSS.Bucket
			err = setBucketLogging(osscli, bucketName)

View on GitHub (pinned to 35bff19146)

Solutions

1. Check the wrapped error for the failing object; grant GetObject on all keys under the prefix.
2. Verify the pod has enough disk space in the destination volume for the entire directory.
3. If only some files are needed, reference the specific object keys individually instead of the directory prefix.
4. Re-run the workflow if objects were concurrently deleted/overwritten mid-download.

Example fix

// before: downloading whole prefix with partial permissions
oss:
  bucket: b
  key: outputs/
// after: grant policy on prefix
{ "Action": ["oss:GetObject"], "Resource": ["acs:oss:*:*:b/outputs/*"], "Effect": "Allow" }

// preflight: enumerate the prefix and verify overall size fits the destination volume
b, _ := cli.Bucket(bucket)
ls, err := b.ListObjects(oss.Prefix(key))
if err != nil {
	return err
}
var total int64
for _, o := range ls.Objects {
	total += o.Size
}
// compare total against available space on the target mount before Load

err := driver.Load(ctx, art, path)
if err != nil && strings.Contains(err.Error(), "failed get directory") {
	// recursive download failed mid-way; log wrapped cause, retry or shrink scope
}

When it happens

Trigger: Artifact key references a folder-style prefix that exists in OSS, GetOssDirectory enumerates its objects, and one or more sub-operations fail: credentials can List but not Get some objects, an object disappears mid-download, or a local disk write fails on a large directory.

Common situations: Downloading folder outputs produced by a previous workflow step where the RAM policy only covers a subset of keys; huge directories exceeding disk space in the pod; concurrent modification of the prefix while downloading.

Related errors

• mkdir %s error: %w
• failed to get file: %w
• failed to test if %s/%s is a directory: %w
• failed to get file: %w
• unable to download directory %s: %w

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/168f3344b55347ec. Report an issue: GitHub.