{"record":{"id":"b431b6f222fc53c7","repo":"argoproj/argo-workflows","slug":"upload-s-w","errorCode":null,"errorMessage":"upload %s: %w","messagePattern":"upload (.+?): %w","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"workflow/artifacts/gcs/gcs.go","lineNumber":296,"sourceCode":"\n// upload a local file or dir to GCS\nfunc uploadObjects(ctx context.Context, client *storage.Client, bucket, key, path string) error {\n\tisDir, err := file.IsDirectory(path)\n\tif err != nil {\n\t\treturn fmt.Errorf(\"test if %s is a dir: %w\", path, err)\n\t}\n\tif isDir {\n\t\tdirName := filepath.Clean(path) + string(os.PathSeparator)\n\t\tkeyPrefix := filepath.Clean(key) + \"/\"\n\t\tfileRelPaths, listErr := listFileRelPaths(dirName, \"\")\n\t\tif listErr != nil {\n\t\t\treturn listErr\n\t\t}\n\t\tfor _, relPath := range fileRelPaths {\n\t\t\tfullKey := normalizeGCSKey(keyPrefix + relPath)\n\t\t\terr = uploadObject(ctx, client, bucket, fullKey, dirName+relPath)\n\t\t\tif err != nil {\n\t\t\t\treturn fmt.Errorf(\"upload %s: %w\", dirName+relPath, err)\n\t\t\t}\n\t\t}\n\t} else {\n\t\tobjectKey := normalizeGCSKey(filepath.Clean(key))\n\t\terr = uploadObject(ctx, client, bucket, objectKey, path)\n\t\tif err != nil {\n\t\t\treturn fmt.Errorf(\"upload %s: %w\", path, err)\n\t\t}\n\t}\n\treturn nil\n}\n\n// upload an object to GCS\nfunc uploadObject(ctx context.Context, client *storage.Client, bucket, key, localPath string) error {\n\tf, err := os.Open(filepath.Clean(localPath))\n\tif err != nil {\n\t\treturn fmt.Errorf(\"os open: %w\", err)\n\t}","sourceCodeStart":278,"sourceCodeEnd":314,"githubUrl":"https://github.com/argoproj/argo-workflows/blob/35bff19146f5a6ada77468c431f2624bd577e373/workflow/artifacts/gcs/gcs.go#L278-L314","documentation":"Once the source is known to be a directory, uploadObjects uploads each contained file with uploadObject, wrapping per-file failures as \"upload <path>\". This error means one specific file inside the artifact directory failed to upload to GCS; the wrapped cause distinguishes the GCS API error (permission, quota, network) from local read errors.","triggerScenarios":"uploadObject fails for one file of a directory artifact: writer creation rejected by GCS (no storage.objects.create permission, bucket policy/quota), local file unreadable mid-listing (deleted between list and upload), or network failure during the upload stream.","commonSituations":"GSA missing roles/storage.objectCreator on the bucket; bucket CMEK/key permissions missing; huge output directories where one unreadable file (e.g. dangling symlink content, /proc-like entry) aborts the whole artifact save; transient 5xx from GCS.","solutions":["Grant the credential roles/storage.objectCreator (or objectAdmin) on the bucket","Check bucket encryption (CMEK) key permissions for the upload identity","Retry the workflow if the wrapped error is transient (5xx/network)","Inspect the directory for unreadable/special files and exclude them from the artifact path"],"exampleFix":"// before\ngcloud projects remove-iam-policy-binding proj --member=serviceAccount:wf-sa@proj.iam --role=roles/storage.objectCreator\n// after\ngcloud projects add-iam-policy-binding proj --member=serviceAccount:wf-sa@proj.iam --role=roles/storage.objectCreator","handlingStrategy":"retry","validationCode":"// pre-flight write permission with the same identity:\n// gcloud storage cp - / gs://my-bucket/.perm-check <<< \"ok\"\n// fails with 403 if the identity lacks storage.objects.create","typeGuard":null,"tryCatchPattern":"err := uploadArtifacts(ctx, dir)\nif err != nil {\n\tif strings.Contains(err.Error(), \"upload \") && utilerrors.IsTransientErr(ctx, err) {\n\t\treturn retryWithBackoff(err) // 5xx/network per-file failure\n\t}\n\treturn err // 403/quota: fix IAM, not retry\n}","preventionTips":["Grant roles/storage.objectCreator on the artifact bucket to the workflow identity","Exclude transient/junk files from output directories before saving artifacts","Check bucket quotas and CMEK key IAM if uploads fail with encryption errors"],"tags":["gcs","upload","permissions","artifacts"],"backgroundTag":"gcs-object-write-failed","analyzedSha":"35bff19146f5a6ada77468c431f2624bd577e373","analyzedAt":"2026-09-03T19:34:35.908Z","contentChangedAt":"2026-09-03T19:34:35.908Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}