VictoriaMetrics/VictoriaMetrics · error

cannot create HTTP client for %q: %w

Error message

cannot create HTTP client for %q: %w

What it means

This error wraps any failure from discoveryutil.NewClient when the http_sd discovery mechanism builds its HTTP client. NewClient validates the auth config (TLS, basic auth, OAuth2, headers) and proxy settings against the parsed apiServer host; if any of those cannot be turned into a usable HTTP client, the error is wrapped with the apiServer URL so the user knows which endpoint config is broken. It occurs at scrape-config start time, before any actual network requests are made.

Source

Thrown at lib/promscrape/discovery/http/api.go:65

func newAPIConfig(sdc *SDConfig, baseDir string) (*apiConfig, error) {
	ac, err := sdc.HTTPClientConfig.NewConfig(baseDir)
	if err != nil {
		return nil, fmt.Errorf("cannot parse auth config: %w", err)
	}
	parsedURL, err := url.Parse(sdc.URL)
	if err != nil {
		return nil, fmt.Errorf("cannot parse http_sd URL: %w", err)
	}
	apiServer := fmt.Sprintf("%s://%s", parsedURL.Scheme, parsedURL.Host)

	proxyAC, err := sdc.ProxyClientConfig.NewConfig(baseDir)
	if err != nil {
		return nil, fmt.Errorf("cannot parse proxy auth config: %w", err)
	}
	client, err := discoveryutil.NewClient(apiServer, ac, sdc.ProxyURL, proxyAC, &sdc.HTTPClientConfig)
	if err != nil {
		return nil, fmt.Errorf("cannot create HTTP client for %q: %w", apiServer, err)
	}
	cfg := &apiConfig{
		client:        client,
		path:          parsedURL.RequestURI(),
		sourceURL:     sdc.URL,
		checkInterval: max(*SDCheckInterval/2, time.Second),
		fetchErrors:   metrics.GetOrCreateCounter(fmt.Sprintf(`promscrape_discovery_http_errors_total{type="fetch",url=%q}`, sdc.URL)),
		parseErrors:   metrics.GetOrCreateCounter(fmt.Sprintf(`promscrape_discovery_http_errors_total{type="parse",url=%q}`, sdc.URL)),
	}
	cfg.wg.Go(func() {
		cfg.run()
	})
	return cfg, nil
}

func (cfg *apiConfig) init() {
	cfg.initOnce.Do(func() {
		cfg.refreshTargetsIfNeeded()

View on GitHub (pinned to 5079fb58f1)

Solutions

1. Check the wrapped inner error for the exact cause; verify all files referenced by http_sd_config (tls_config cert/key/ca, bearer_token_file, credentials) exist and are readable by the VictoriaMetrics process
2. Fix paths relative to the config's baseDir and confirm cert/key pairs match and are valid PEM
3. Remove or correct unsupported/invalid fields in http_sd_config (e.g. incomplete oauth2 block) and restart vmagent

Example fix

// before
- http_sd_config:
  - url: http://sd-api/targets
    tls_config:
      cert_file: /etc/vmagent/client.crt
      key_file: /etc/vmagent/client.key
// after
- http_sd_config:
  - url: http://sd-api/targets
    tls_config:
      cert_file: /etc/vmagent/tls/client.crt   # file exists and is readable
      key_file: /etc/vmagent/tls/client.key    # matches client.crt

files := []string{tlsCfg.CertFile, tlsCfg.KeyFile, tlsCfg.CAFile, authCfg.BearerTokenFile}
for _, f := range files {
	if f != "" {
		if _, err := os.Stat(f); err != nil {
			return fmt.Errorf("http_sd auth file %q unreadable: %w", f, err)
		}
	}
}

cfg, err := sdc.MustStart(baseDir) // check sdc.startErr / wrapped error before relying on targets
if err != nil {
	log.Errorf("http_sd init failed: %v", err)
}

When it happens

Trigger: Calling SDConfig.MustStart(baseDir) with an http_sd_config whose HTTPClientConfig (TLS, basic_auth, oauth2) or ProxyClientConfig contains invalid settings, e.g. TLS certificate files that do not exist or cannot be read, malformed cert/key pairs, or an OAuth2 config missing required fields — NewClient returns an error which is wrapped here.

Common situations: Users point tls_config cert_file/key_file at paths that don't exist relative to baseDir, mount secrets after vmagent starts, use an expired or unreadable bearer token file, or copy a Prometheus http_sd_config with OAuth2 fields unsupported/incomplete in VictoriaMetrics.

Related errors

• cannot create HTTP client for %q: %w
• identity_endpoint v2.0 is not supported
• URL is missing
• query missing
• cannot gracefully shutdown http server at %q in %.3fs; proba

AI-assisted analysis of VictoriaMetrics/VictoriaMetrics@5079fb58f1 (2026-09-03). Data as JSON: /api/errors/b07bcd71753c1d94. Report an issue: GitHub.