{"record":{"id":"1b34208a98d20fca","repo":"argoproj/argo-workflows","slug":"unable-to-process-data-source-w","errorCode":null,"errorMessage":"unable to process data source: %w","messagePattern":"unable to process data source: %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"workflow/data/data.go","lineNumber":15,"sourceCode":"package data\n\nimport (\n\t\"context\"\n\t\"fmt\"\n\n\t\"github.com/expr-lang/expr\"\n\n\twfv1 \"github.com/argoproj/argo-workflows/v4/pkg/apis/workflow/v1alpha1\"\n)\n\nfunc ProcessData(ctx context.Context, data *wfv1.Data, processor wfv1.DataSourceProcessor) (any, error) {\n\tsourcedData, err := processSource(ctx, data.Source, processor)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"unable to process data source: %w\", err)\n\t}\n\ttransformedData, err := processTransformation(sourcedData, &data.Transformation)\n\tif err != nil {\n\t\treturn nil, fmt.Errorf(\"unable to process data transformation: %w\", err)\n\t}\n\treturn transformedData, nil\n}\n\nfunc processSource(ctx context.Context, source wfv1.DataSource, processor wfv1.DataSourceProcessor) (any, error) {\n\tvar data any\n\tvar err error\n\tswitch {\n\tcase source.ArtifactPaths != nil:\n\t\tdata, err = processor.ProcessArtifactPaths(ctx, source.ArtifactPaths)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"unable to source artifact paths: %w\", err)\n\t\t}\n\tdefault:","sourceCodeStart":1,"sourceCodeEnd":33,"githubUrl":"https://github.com/argoproj/argo-workflows/blob/35bff19146f5a6ada77468c431f2624bd577e373/workflow/data/data.go#L1-L33","documentation":"Argo's ProcessData failed at the first stage of a Data template: retrieving data from the configured source (currently only artifactPaths). The underlying processor error (e.g. listing artifacts from S3/GCS) is wrapped with this message. A Data template cannot proceed to transformation if the source stage fails.","triggerScenarios":"A Workflow template of type Data runs; processor.ProcessArtifactPaths errors while listing/matching artifact paths in the artifact repository (bad bucket, wrong key prefix, credentials, unreachable storage).","commonSituations":"Misconfigured artifact repository credentials, artifactPaths glob matching nothing or a nonexistent prefix, storage backend outage, or artifact driver misconfiguration in the controller configmap.","solutions":["Look at the wrapped cause in the controller log for the storage-level error (auth, not-found, timeout)","Verify artifact repository config (configmap workflow-controller-configmap artifactRepository) and credentials/keys","Test the artifactPaths prefix/glob against your actual artifact layout — ensure artifacts exist under that key prefix","Run a simpler Artifact-template on the same storage to isolate driver vs data-source issues"],"exampleFix":"// before\nsource:\n  artifactPaths:\n    s3:\n      key: wrong-prefix/outcomes/*.tgz   // no artifacts here\n// after\nsource:\n  artifactPaths:\n    s3:\n      key: my-bucket/artifacts/{{workflow.name}}/*.tgz","handlingStrategy":"validation","validationCode":"// Validate the data source before submitting\nd := wf.Spec.Templates[i].Data\nif d == nil || d.Source.ArtifactPaths == nil {\n    return errors.New(\"data template requires source.artifactPaths\")\n}\n// and confirm artifacts exist under the prefix\nout, err := exec.Command(\"aws\", \"s3\", \"ls\", keyPrefix).Output()\nif err != nil || len(out) == 0 { return errors.New(\"no artifacts under key prefix\") }","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Verify artifact repository credentials and config in the controller configmap","Confirm artifacts exist under the artifactPaths prefix with a storage CLI first","Lint workflows with argo lint before submission","Test storage connectivity with a minimal artifact template"],"tags":["data-template","artifacts","storage","argo-workflows"],"backgroundTag":"artifact-source-processing-failed","analyzedSha":"35bff19146f5a6ada77468c431f2624bd577e373","analyzedAt":"2026-09-03T19:34:35.908Z","contentChangedAt":"2026-09-03T19:34:35.908Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}