VictoriaMetrics/VictoriaMetrics · error

cannot get API config for kuma_sd: %w

Error message

cannot get API config for kuma_sd: %w

What it means

SDConfig.GetLabels resolves the shared kuma_sd apiConfig via getAPIConfig -> newAPIConfig. Since the initial targets fetch happens synchronously inside newAPIConfig, any configuration error (bad server URL, invalid auth config, HTTP client creation failure) or initial discovery failure is wrapped here. The scrape config referencing this kuma_sd_config is rejected and target discovery for that job does not start.

Source

Thrown at lib/promscrape/discovery/kuma/kuma.go:40

	Server   string `yaml:"server"`
	ClientID string `yaml:"client_id,omitempty"`

	HTTPClientConfig  promauth.HTTPClientConfig  `yaml:",inline"`
	ProxyURL          *proxy.URL                 `yaml:"proxy_url,omitempty"`
	ProxyClientConfig promauth.ProxyClientConfig `yaml:",inline"`

	// fetch_timeout isn't used, so it isn't defined.
	// FetchTimeout time.Duration `yaml:"fetch_timeout,omitempty"`

	// refresh_interval is obtained from `-promscrape.kumaSDCheckInterval` command-line option.
	// RefreshInterval time.Duration `yaml:"refresh_interval,omitempty"`
}

// GetLabels returns kuma service discovery labels according to sdc.
func (sdc *SDConfig) GetLabels(baseDir string) ([]*promutil.Labels, error) {
	cfg, err := getAPIConfig(sdc, baseDir)
	if err != nil {
		return nil, fmt.Errorf("cannot get API config for kuma_sd: %w", err)
	}
	pLabels := cfg.labels.Load()
	return *pLabels, nil
}

// MustStop stops further usage for sdc.
func (sdc *SDConfig) MustStop() {
	v := configMap.Delete(sdc)
	if v != nil {
		// v can be nil if GetLabels wasn't called yet.
		cfg := v.(*apiConfig)
		cfg.mustStop()
	}
}

View on GitHub (pinned to 5079fb58f1)

Solutions

1. Read the wrapped cause (%w) below this message in the log to identify the failing sub-step (server parse, auth config, client creation, or initial fetch).
2. Ensure kuma_sd_config has a valid server URL, e.g. 'http://kuma-cp.kuma-system.svc:5678'.
3. Verify all files referenced by http_client_config (ca, cert, key, bearer_token_file) exist and are readable before starting VictoriaMetrics.
4. Make the Kuma control plane reachable before starting VictoriaMetrics, since the initial discovery fetch is synchronous and aborts config load on failure.
5. Validate the scrape config with the -promscrape.config check mode before restarting.

Example fix

// before
kuma_sd_configs:
  - server: ''   # empty server -> 'cannot parse server' -> wrapped here
// after
kuma_sd_configs:
  - server: 'http://kuma-cp.kuma-system.svc:5678'

# Validate the scrape config before restart:
/victoria-metrics -promscrape.config=/etc/vm/promscrape.yml -promscrape.config.dryRun
# Ensure the kuma_sd_config has a non-empty, reachable server:
curl -sf http://kuma-cp.kuma-system.svc:5678/v3/discovery:monitoringassignments -o /dev/null && echo OK

When it happens

Trigger: GetLabels is called when a scrape config with kuma_sd_configs is loaded. getAPIConfig -> newAPIConfig fails: getAPIServerPath rejects an empty/invalid server URL; HTTPClientConfig.NewConfig or ProxyClientConfig.NewConfig fails (missing cert files, bad basicauth); discoveryutil.NewClient fails; or the initial cfg.updateTargetsLabels fails (network unreachable, parse error) causing 'cannot discover Kuma targets'.

Common situations: Kuma section omitted 'server' entirely; typo in server URL; referenced tls_config cert files don't exist at startup (paths relative to config baseDir); Kuma control plane unreachable when VM/vmagent boots, so startup fails instead of retrying later.

Related errors

• cannot discover Kuma targets: %w
• cannot create API config for http_sd: %w
• cannot parse server %q: %w
• cannot create HTTP client for %q: %w
• missing server url

AI-assisted analysis of VictoriaMetrics/VictoriaMetrics@5079fb58f1 (2026-09-03). Data as JSON: /api/errors/886615a846fb4f17. Report an issue: GitHub.