argoproj/argo-workflows · error

'resourcesDuration.*' metrics cannot be used in real-time

Error message

'resourcesDuration.*' metrics cannot be used in real-time

What it means

Template validation guard: a realtime gauge (gauge.realtime: true) references a resourcesDuration.* variable. Realtime gauges are evaluated continuously and cannot consume the final resource-duration metrics, which only exist after node completion.

Source

Thrown at workflow/metrics/util.go:30

var (
	invalidMetricNameError  = "metric name is invalid: names may only contain alphanumeric characters or '_'"
	invalidMetricLabelError = "metric label '%s' is invalid: keys may only contain alphanumeric characters or '_'"
)

func IsValidMetricName(name string) bool {
	// Use promtheus's metric name checker, despite perhaps not using prometheus
	return model.LegacyValidation.IsValidMetricName(string(model.LabelValue(name))) && !strings.Contains(name, `:`)
}

func ValidateMetricValues(metric *wfv1.Prometheus) error {
	if metric.Gauge != nil {
		if metric.Gauge.Value == "" {
			return errors.New("missing gauge.value")
		}
		if metric.Gauge.Realtime != nil && *metric.Gauge.Realtime {
			if strings.Contains(metric.Gauge.Value, "resourcesDuration.") {
				return errors.New("'resourcesDuration.*' metrics cannot be used in real-time")
			}
		}
	}
	if metric.Counter != nil && metric.Counter.Value == "" {
		return errors.New("missing counter.value")
	}
	if metric.Histogram != nil && metric.Histogram.Value == "" {
		return errors.New("missing histogram.value")
	}
	return nil
}

func ValidateMetricLabels(metrics map[string]string) error {
	for name := range metrics {
		if !IsValidMetricName(name) {
			return fmt.Errorf(invalidMetricLabelError, name)
		}
	}

View on GitHub (pinned to 35bff19146)

Solutions

1. Remove realtime: true from the gauge, or
2. Use a different metric value that is available during execution

When it happens

Trigger: Thrown at workflow/metrics/util.go:30 when the library encounters an invalid state.

Common situations: See trigger scenarios.

AI-assisted analysis of argoproj/argo-workflows@35bff19146 (2026-09-03). Data as JSON: /api/errors/61cfa2c12c7c623c. Report an issue: GitHub.