argoproj/argo-workflows · error

invalid custom metric type

Error message

invalid custom metric type

What it means

UpsertCustomMetric switches on the metricSpec shape (Gauge/Counter/Histogram/realtime); if the spec has none of the recognized metric kinds set — its GetMetricType() is unsupported — the default branch returns 'invalid custom metric type'. The Prometheus spec in the workflow must declare exactly one of gauge, counter, or histogram.

Source

Thrown at workflow/metrics/metrics_custom.go:258

			fallthrough
		default:
			metricValue.prometheusValue = val
		}
	case metricType == wfv1.MetricTypeHistogram:
		val, err := strconv.ParseFloat(metricSpec.Histogram.Value, 64)
		if err != nil {
			return err
		}
		// Record does its own locking if needed by the exporter
		baseMetric.Record(ctx, val, metricValue.getLabels())
	case metricType == wfv1.MetricTypeCounter:
		val, err := strconv.ParseFloat(metricSpec.Counter.Value, 64)
		if err != nil {
			return err
		}
		metricValue.prometheusValue += val
	default:
		return fmt.Errorf("invalid custom metric type")
	}
	return nil
}

func (m *Metrics) attachCustomMetricToWorkflow(metricSpec *wfv1.Prometheus, ownerKey string) {
	if !metricSpec.IsRealtime() {
		return
	}
	m.realtimeMutex.Lock()
	defer m.realtimeMutex.Unlock()
	metricKey := metricSpec.GetKey()
	for _, tracker := range m.realtimeWorkflows[ownerKey] {
		if tracker.key == metricKey {
			return
		}
	}
	m.realtimeWorkflows[ownerKey] = append(m.realtimeWorkflows[ownerKey], realtimeTracker{
		inst: m.GetInstrument(metricSpec.Name),

View on GitHub (pinned to 35bff19146)

Solutions

1. Add exactly one metric type block (gauge:, counter:, or histogram:) to the metric spec
2. Fix YAML key typos so the intended type block actually parses (argo lint helps)
3. Ensure the spec is constructed with the correct field when building Prometheus objects in code

Example fix

# before
metrics:
  - name: my_metric
    value: "1"
# after
metrics:
  - name: my_metric
    gauge:
      value: "1"

// ensure the spec declares exactly one metric type
func hasMetricType(m *wfv1.Prometheus) bool {
	return m.GetMetricType() != "" && (m.Gauge != nil || m.Counter != nil || m.Histogram != nil)
}

When it happens

Trigger: Submitting/upserting a metric whose wfv1.Prometheus spec has no gauge/counter/histogram stanza (all nil), or an empty/zero-value Prometheus entry in the workflow's metrics list.

Common situations: Hand-edited workflow YAML where the type block was deleted or mis-indented; programmatically built specs leaving the type field unset; typos in the YAML key (e.g. 'guage') that silently parse as an unknown field.

Related errors

• invalid metric spec
• CodeBadRequest
• successCondition, failureCondition and outputs are not suppo
• %s
• output parameters must have a valueFrom specified

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/b9613531bd893660. Report an issue: GitHub.