argoproj/argo-workflows · error

failed to get directory: %w

Error message

failed to get directory: %w

What it means

After NoSuchKey and a successful IsDirectory=true probe, the driver downloads the key as a directory (all objects under the prefix). This error wraps a failure of GetDirectory — the directory download itself failed after the key was confirmed to be a prefix.

Source

Thrown at workflow/artifacts/s3/s3.go:227

	origErr := s3cli.GetFile(inputArtifact.S3.Bucket, inputArtifact.S3.Key, path)
	if origErr == nil {
		return true, nil
	}
	if !IsS3ErrCode(origErr, "NoSuchKey") {
		return !isTransientS3Err(ctx, origErr), fmt.Errorf("failed to get file: %w", origErr)
	}
	// If we get here, the error was a NoSuchKey. The key might be an s3 "directory"
	isDir, err := s3cli.IsDirectory(inputArtifact.S3.Bucket, inputArtifact.S3.Key)
	if err != nil {
		return !isTransientS3Err(ctx, err), fmt.Errorf("failed to test if %s is a directory: %w", inputArtifact.S3.Key, err)
	}
	if !isDir {
		// It's neither a file, nor a directory. Return the original NoSuchKey error
		return true, argoerrs.New(argoerrs.CodeNotFound, origErr.Error())
	}

	if err = s3cli.GetDirectory(inputArtifact.S3.Bucket, inputArtifact.S3.Key, path); err != nil {
		return !isTransientS3Err(ctx, err), fmt.Errorf("failed to get directory: %w", err)
	}
	return true, nil
}

// OpenStream opens a stream reader for an artifact from S3 compliant storage
func (s3Driver *ArtifactDriver) OpenStream(ctx context.Context, inputArtifact *wfv1.Artifact) (io.ReadCloser, error) {
	log := logging.RequireLoggerFromContext(ctx)
	log.WithField("key", inputArtifact.S3.Key).Info(ctx, "S3 OpenStream")
	//nolint:contextcheck
	s3cli, err := s3Driver.newClient(log.NewBackgroundContext())
	if err != nil {
		return nil, fmt.Errorf("failed to create new S3 client: %w", err)
	}

	return streamS3Artifact(ctx, s3cli, inputArtifact)
}

func streamS3Artifact(_ context.Context, s3cli Client, inputArtifact *wfv1.Artifact) (io.ReadCloser, error) {

View on GitHub (pinned to 35bff19146)

Solutions

1. Retry the workflow — the driver marks transient errors retryable via isTransientS3Err/backoff
2. Check pod disk space and volume mounts where the artifact is extracted
3. Verify s3:GetObject permission on all objects under the key prefix
4. Check bucket endpoint/region configuration and S3 provider health; test listing the prefix manually with aws s3 cp

aws s3 cp --recursive s3://my-bucket/my/key/ /tmp/test >/dev/null && echo OK

if err := s3Driver.Load(ctx, path, art); err != nil {
    if errors.Is(err, context.DeadlineExceeded) || isTransientS3Err(ctx, err) {
        // safe to retry
    }
}

When it happens

Trigger: loadS3Artifact: s3cli.GetDirectory(bucket, key, path) returned an error while listing/downloading all objects under the prefix — partial download failure, network interruption, permission denied on individual objects, or disk write failure at `path`.

Common situations: Large directory artifacts interrupted mid-transfer; IAM allows ListBucket but denies GetObject on some objects; local volume full in the workflow pod; S3-compatible storage returning malformed listing responses.

Related errors

• failed to test if %s is a directory: %w
• failed to put directory: %w
• failed to put file: %w
• failed to check if key %s exists from bucket %s: %w
• Failed to save artifact: %v

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/55a5a0cce1024fce. Report an issue: GitHub.