argoproj/argo-workflows · error

unable to process data source: %w

Error message

unable to process data source: %w

What it means

Argo's ProcessData failed at the first stage of a Data template: retrieving data from the configured source (currently only artifactPaths). The underlying processor error (e.g. listing artifacts from S3/GCS) is wrapped with this message. A Data template cannot proceed to transformation if the source stage fails.

Source

Thrown at workflow/data/data.go:15

package data

import (
	"context"
	"fmt"

	"github.com/expr-lang/expr"

	wfv1 "github.com/argoproj/argo-workflows/v4/pkg/apis/workflow/v1alpha1"
)

func ProcessData(ctx context.Context, data *wfv1.Data, processor wfv1.DataSourceProcessor) (any, error) {
	sourcedData, err := processSource(ctx, data.Source, processor)
	if err != nil {
		return nil, fmt.Errorf("unable to process data source: %w", err)
	}
	transformedData, err := processTransformation(sourcedData, &data.Transformation)
	if err != nil {
		return nil, fmt.Errorf("unable to process data transformation: %w", err)
	}
	return transformedData, nil
}

func processSource(ctx context.Context, source wfv1.DataSource, processor wfv1.DataSourceProcessor) (any, error) {
	var data any
	var err error
	switch {
	case source.ArtifactPaths != nil:
		data, err = processor.ProcessArtifactPaths(ctx, source.ArtifactPaths)
		if err != nil {
			return nil, fmt.Errorf("unable to source artifact paths: %w", err)
		}
	default:

View on GitHub (pinned to 35bff19146)

Solutions

1. Look at the wrapped cause in the controller log for the storage-level error (auth, not-found, timeout)
2. Verify artifact repository config (configmap workflow-controller-configmap artifactRepository) and credentials/keys
3. Test the artifactPaths prefix/glob against your actual artifact layout — ensure artifacts exist under that key prefix
4. Run a simpler Artifact-template on the same storage to isolate driver vs data-source issues

Example fix

// before
source:
  artifactPaths:
    s3:
      key: wrong-prefix/outcomes/*.tgz   // no artifacts here
// after
source:
  artifactPaths:
    s3:
      key: my-bucket/artifacts/{{workflow.name}}/*.tgz

// Validate the data source before submitting
d := wf.Spec.Templates[i].Data
if d == nil || d.Source.ArtifactPaths == nil {
    return errors.New("data template requires source.artifactPaths")
}
// and confirm artifacts exist under the prefix
out, err := exec.Command("aws", "s3", "ls", keyPrefix).Output()
if err != nil || len(out) == 0 { return errors.New("no artifacts under key prefix") }

When it happens

Trigger: A Workflow template of type Data runs; processor.ProcessArtifactPaths errors while listing/matching artifact paths in the artifact repository (bad bucket, wrong key prefix, credentials, unreachable storage).

Common situations: Misconfigured artifact repository credentials, artifactPaths glob matching nothing or a nonexistent prefix, storage backend outage, or artifact driver misconfiguration in the controller configmap.

Related errors

• unable to source artifact paths: %w
• error appending filename %s to key of artifact %+v: err: %w
• Artifact driver connection validation failed: %v
• error getting key for artifact: %w
• failed to re-enter working directory %q after staging input

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/1b34208a98d20fca. Report an issue: GitHub.