{"record":{"id":"cd662f775cecc8a6","repo":"argoproj/argo-workflows","slug":"help-for-metric-s-is-already-set-to-s-it-cannot","errorCode":null,"errorMessage":"help for metric %s is already set to %s, it cannot be changed","messagePattern":"help for metric (.+?) is already set to (.+?), it cannot be changed","errorType":"validation","errorClass":null,"httpStatus":null,"severity":"error","filePath":"workflow/metrics/metrics_custom.go","lineNumber":144,"sourceCode":"// GetCustomMetric returns a custom (or any) metric from it's key\n// This is exported for legacy testing only\nfunc (m *Metrics) GetCustomMetric(key string) *telemetry.Instrument {\n\t// It's okay to return nil metrics in this function\n\treturn m.GetInstrument(key)\n}\n\n// CustomMetricExists returns if metric exists from its key\n// This is exported for testing only\nfunc (m *Metrics) CustomMetricExists(key string) bool {\n\treturn m.GetCustomMetric(key) != nil\n}\n\n// TODO labels on custom metrics\nfunc (m *Metrics) matchExistingMetric(metricSpec *wfv1.Prometheus) (*telemetry.Instrument, error) {\n\tkey := metricSpec.Name\n\tif inst := m.GetInstrument(key); inst != nil {\n\t\tif inst.GetDescription() != metricSpec.Help {\n\t\t\treturn nil, fmt.Errorf(\"help for metric %s is already set to %s, it cannot be changed\", metricSpec.Name, inst.GetDescription())\n\t\t}\n\t\twantedType := metricSpec.GetMetricType()\n\t\tswitch inst.GetOtel().(type) {\n\t\tcase *metric.Float64ObservableGauge:\n\t\t\tif wantedType != wfv1.MetricTypeGauge && !metricSpec.IsRealtime() {\n\t\t\t\treturn nil, fmt.Errorf(\"found existing gauge for custom metric %s of type %s\", metricSpec.Name, wantedType)\n\t\t\t}\n\t\tcase *metric.Float64ObservableCounter:\n\t\t\tif wantedType != wfv1.MetricTypeCounter {\n\t\t\t\treturn nil, fmt.Errorf(\"found existing counter for custom metric %s of type %s\", metricSpec.Name, wantedType)\n\t\t\t}\n\t\tcase *metric.Float64Histogram:\n\t\t\tif wantedType != wfv1.MetricTypeHistogram {\n\t\t\t\treturn nil, fmt.Errorf(\"found existing histogram for custom metric %s of type %s\", metricSpec.Name, wantedType)\n\t\t\t}\n\t\tdefault:\n\t\t\treturn nil, fmt.Errorf(\"found unwanted type %s for custom metric %s of type %s\", reflect.TypeOf(inst.GetOtel()), metricSpec.Name, wantedType)\n\t\t}","sourceCodeStart":126,"sourceCodeEnd":162,"githubUrl":"https://github.com/argoproj/argo-workflows/blob/35bff19146f5a6ada77468c431f2624bd577e373/workflow/metrics/metrics_custom.go#L126-L162","documentation":"When a workflow (or a retry/resubmit of it) re-declares a Prometheus custom metric whose name already exists, matchExistingMetric verifies the new spec is consistent with the already-registered OpenTelemetry instrument. If metricSpec.Help differs from the existing instrument's description, the help text of a registered metric cannot be changed, so the executor fails the metric (and the workflow is marked Error).","triggerScenarios":"A workflow spec defines two Prometheus metrics with the same `name` but different `help` strings, or a retried/resubmitted workflow changes the help text of a metric name that was already registered in the executor's meter.","commonSituations":"Copy-pasting a metric block and editing help but not name; retrying a workflow whose updated spec tweaked the help string; templates parameterized such that help text varies per step while name stays constant.","solutions":["Give each distinct help text a distinct metric name","Make the help string identical across all declarations of the same metric name (including retries of prior runs)","If the change was intentional, run in a fresh executor pod / workflow name so the instrument is created anew","Check metric help templating (e.g. {{workflow.name}}) isn't producing varying strings"],"exampleFix":"// before\n- name: duration\n  help: How long the step took\n- name: duration\n  help: Step duration  # mismatch\n// after\n- name: duration\n  help: How long the step took\n- name: duration_steps\n  help: Step duration\n","handlingStrategy":"validation","validationCode":"// ensure help text is constant per metric name across the spec\nhelpByName := map[string]string{}\nfor _, m := range spec.Prometheus {\n    if h, seen := helpByName[m.Name]; seen && h != m.Help {\n        return fmt.Errorf(\"metric %s has conflicting help: %q vs %q\", m.Name, h, m.Help)\n    }\n    helpByName[m.Name] = m.Help\n}","typeGuard":null,"tryCatchPattern":"if err != nil && strings.Contains(err.Error(), \"is already set to\") {\n    // fix spec help/name mismatch and resubmit as a new workflow\n}","preventionTips":["Never vary `help` for the same metric name, including across retries/resubmits","Use unique metric names for metrics with different descriptions","Beware templated help strings that resolve differently per step","Run `argo lint` before submit"],"tags":["metrics","prometheus","otel","workflow"],"backgroundTag":"metric-help-conflict","analyzedSha":"35bff19146f5a6ada77468c431f2624bd577e373","analyzedAt":"2026-09-03T19:34:35.908Z","contentChangedAt":"2026-09-03T19:34:35.908Z","schemaVersion":2},"datasetVersion":"2026-09-08T10:18:20.063Z"}