VictoriaMetrics/VictoriaMetrics · error

cannot query %q: %w

Error message

cannot query %q: %w

What it means

getAPIResponse performs a plain client.Get(apiURL) against the GCE Compute API (with filter and pageToken query args). If the HTTP request itself fails — DNS failure, TLS error, connection refused/timeout — the URL is wrapped in this message. This fires before any response-body reading, so it signals transport-level failure, not an API error status.

Source

Thrown at lib/promscrape/discovery/gce/api.go:96

	if sdc.Port != nil {
		port = *sdc.Port
	}
	return &apiConfig{
		client:       client,
		zones:        zones,
		project:      project,
		filter:       sdc.Filter,
		tagSeparator: tagSeparator,
		port:         port,
	}, nil
}

func getAPIResponse(client *http.Client, apiURL, filter, pageToken string) ([]byte, error) {
	apiURL = appendNonEmptyQueryArg(apiURL, "filter", filter)
	apiURL = appendNonEmptyQueryArg(apiURL, "pageToken", pageToken)
	resp, err := client.Get(apiURL)
	if err != nil {
		return nil, fmt.Errorf("cannot query %q: %w", apiURL, err)
	}
	return readResponseBody(resp, apiURL)
}

func readResponseBody(resp *http.Response, apiURL string) ([]byte, error) {
	data, err := io.ReadAll(resp.Body)
	_ = resp.Body.Close()
	if err != nil {
		return nil, fmt.Errorf("cannot read response from %q: %w", apiURL, err)
	}
	if resp.StatusCode != http.StatusOK {
		return nil, fmt.Errorf("unexpected status code for %q; got %d; want %d; response body: %q",
			apiURL, resp.StatusCode, http.StatusOK, data)
	}
	return data, nil
}

func appendNonEmptyQueryArg(apiURL, argName, argValue string) string {

View on GitHub (pinned to 5079fb58f1)

Solutions

1. Test connectivity from the vmagent host: curl -v https://compute.googleapis.com/compute/v1/projects/<project>/zones.
2. Enable Private Google Access or a NAT/proxy if the host has no internet egress.
3. If using a TLS-intercepting proxy, add the corporate CA to the system trust store.
4. Fix DNS resolution of compute.googleapis.com (check /etc/resolv.conf, CoreDNS) and retry on transient failures.

Example fix

// before
# VPC without egress to googleapis.com -> connection timeout
// after
# enable Private Google Access on the subnet, or configure HTTPS_PROXY:
$ export HTTPS_PROXY=http://proxy.corp:3128

ctx, cancel := context.WithTimeout(context.Background(), 10*time.Second)
defer cancel()
req, _ := http.NewRequestWithContext(ctx, 'GET', apiURL, nil)
if _, err := http.DefaultClient.Do(req); err != nil {
    return fmt.Errorf('GCE API unreachable before SD start: %w', err)
}

data, err := getAPIResponse(client, apiURL, filter, token)
if err != nil {
    // transport failure: retry with backoff; pageToken makes retries safe
    time.Sleep(backoff)
    data, err = getAPIResponse(client, apiURL, filter, token)
}

When it happens

Trigger: getInstancesForProjectAndZone or getZonesForProject calling getAPIResponse when the GET to compute.googleapis.com fails at the transport layer: no network egress, DNS resolution failure for compute.googleapis.com, TLS interception with untrusted CA, or socket timeout on large page requests.

Common situations: Air-gapped/VPC without internet egress or without Private Google Access; corporate proxy MITM breaking TLS; DNS misconfiguration in containers; transient network blips during paged listing.

Related errors

• cannot obtain response to %q: %w
• error when iterating objects at %q: %w
• no backends found for hostname=%q
• failed to lookupIPAddr for host: %q: %w
• failed to LookupSRV records for host: %q: %w

AI-assisted analysis of VictoriaMetrics/VictoriaMetrics@5079fb58f1 (2026-09-03). Data as JSON: /api/errors/e7fe4f0b11b8c945. Report an issue: GitHub.