{"record":{"id":"c2d6af3dddea28af","repo":"argoproj/argo-workflows","slug":"failed-to-validate-workflow-template-instanceid","errorCode":null,"errorMessage":"failed to validate workflow template instanceid: %w","messagePattern":"failed to validate workflow template instanceid: %w","errorType":"error_code","errorClass":null,"httpStatus":null,"severity":"error","filePath":"server/event/dispatch/operation.go","lineNumber":114,"sourceCode":"\tlogger.WithFields(logging.Fields{\"namespace\": wfeb.Namespace, \"event\": wfeb.Name, \"selector\": selector, \"matched\": matched}).Debug(ctx, \"Selector evaluation\")\n\tsubmit := wfeb.Spec.Submit\n\tif matched && submit != nil {\n\t\t//nolint: contextcheck\n\t\tclient := auth.GetWfClient(o.ctx)\n\t\tref := wfeb.Spec.Submit.WorkflowTemplateRef\n\t\tvar tmpl wfv1.WorkflowSpecHolder\n\t\tvar err error\n\t\tif ref.ClusterScope {\n\t\t\ttmpl, err = client.ArgoprojV1alpha1().ClusterWorkflowTemplates().Get(ctx, ref.Name, metav1.GetOptions{})\n\t\t} else {\n\t\t\ttmpl, err = client.ArgoprojV1alpha1().WorkflowTemplates(wfeb.Namespace).Get(ctx, ref.Name, metav1.GetOptions{})\n\t\t}\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"failed to get workflow template: %w\", err)\n\t\t}\n\t\terr = o.instanceIDService.Validate(tmpl)\n\t\tif err != nil {\n\t\t\treturn nil, fmt.Errorf(\"failed to validate workflow template instanceid: %w\", err)\n\t\t}\n\t\twf := common.NewWorkflowFromWorkflowTemplate(tmpl.GetName(), ref.ClusterScope)\n\n\t\t// Apply workflowMetadata labels and annotations from the template\n\t\t// at creation time, matching the CronWorkflow behavior.\n\t\t// labelsFrom is left to the controller since it\n\t\t// requires parameter evaluation at runtime.\n\t\tif wmd := tmpl.GetWorkflowSpec().WorkflowMetadata; wmd != nil {\n\t\t\tmaps.Copy(wf.Labels, wmd.Labels)\n\t\t\tif len(wmd.Annotations) > 0 {\n\t\t\t\tmaps.Copy(wf.Annotations, wmd.Annotations)\n\t\t\t}\n\t\t}\n\n\t\to.instanceIDService.Label(wf)\n\t\terr = o.populateWorkflowMetadata(wf, &submit.ObjectMeta)\n\t\tif err != nil {\n\t\t\treturn nil, err","sourceCodeStart":96,"sourceCodeEnd":132,"githubUrl":"https://github.com/argoproj/argo-workflows/blob/35bff19146f5a6ada77468c431f2624bd577e373/server/event/dispatch/operation.go#L96-L132","documentation":"After successfully fetching the referenced WorkflowTemplate, the dispatcher validates the template's `metadata.labels[workflows.argoproj.io/instance-id]` against the server's configured instance ID (via instanceid.Service.Validate). If the label is missing or does not match the controller's --instanceid, the template is considered owned by another Argo instance and dispatch is refused. This prevents one Argo installation from submitting workflows from templates it does not manage.","triggerScenarios":"An event matched the binding and the template was fetched, but the template lacks the workflows.argoproj.io/instance-id label while the server runs with --instanceid set, or the label value differs (e.g. template created under a different controller instance or by plain `argo submit` on a differently configured cluster).","commonSituations":"Running multiple Argo controllers/instances against one cluster; templates copied from another namespace/cluster without the instance-id label; templates created manually with kubectl while the server uses a non-empty --instanceid; upgrading or migrating and forgetting to relabel templates.","solutions":["Label the template with the server's instance ID: `kubectl label workflowtemplate <name> -n <ns> workflows.argoproj.io/instance-id=<value>` matching the controller's --instanceid flag.","Or create/manage the template through the same Argo instance (e.g. `argo submit --from workflowtemplate/<name>` via that instance) so the label is applied automatically.","If this Argo instance should accept all templates, unset --instanceid (empty instance ID accepts templates without the label).","Verify with `kubectl get wftmpl <name> -n <ns> --show-labels | grep workflows.argoproj.io/instance-id` that the value matches `kubectl -n argo get deploy workflow-controller -o yaml | grep instanceid`."],"exampleFix":"# before: template missing instance-id label while server runs with --instanceid=prod\n# after\nkubectl label workflowtemplate my-template -n my-ns workflows.argoproj.io/instance-id=prod","handlingStrategy":"validation","validationCode":"# verify instance-id match before dispatch\nSERVER_ID=$(kubectl -n argo get deploy workflow-controller -o jsonpath='{.spec.template.spec.containers[0].args[*]}' | grep -oP '(?<=--instanceid=)\\S+')\nTMPL_ID=$(kubectl get wftmpl <name> -n <ns> -o jsonpath='{.metadata.labels.workflows\\.argoproj\\.io/instance-id}')\n[ \"$SERVER_ID\" = \"$TMPL_ID\" ] || kubectl label wftmpl <name> -n <ns> workflows.argoproj.io/instance-id=$SERVER_ID --overwrite","typeGuard":null,"tryCatchPattern":null,"preventionTips":["Manage WorkflowTemplates through the same Argo instance that dispatches events so the instance-id label is applied automatically.","If running multiple Argo instances, set --instanceid consistently and label templates accordingly.","Keep --instanceid empty if a single instance should own all templates in the cluster.","Add a CI/lint check comparing template instance-id labels with the deployed controller flag."],"tags":["instance-id","configuration","multi-tenancy","argo-workflows","kubernetes"],"backgroundTag":"instance-id-mismatch","analyzedSha":"35bff19146f5a6ada77468c431f2624bd577e373","analyzedAt":"2026-09-03T19:34:35.908Z","contentChangedAt":"2026-09-03T19:34:35.908Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}