wtfutil/wtf · error

%s

Error message

%s

What it means

victorOpsRequest treats any non-200 HTTP status from the VictorOps (Splunk On-Call) API as an error and returns fmt.Errorf("%s", resp.Status), so the message is the raw HTTP status line (e.g. '401 Unauthorized'). The library has no retry or body-based error extraction for this path — the status line is all you get.

Source

Thrown at modules/victorops/client.go:39

func victorOpsRequest(url string, apiID string, apiKey string) ([]OnCallTeam, error) {
	req, err := http.NewRequest("GET", url, http.NoBody)
	if err != nil {
		logger.Log(fmt.Sprintf("Failed to initialize sessions to VictorOps. ERROR: %s", err))
		return nil, err
	}

	req.Header.Set("X-VO-Api-Id", apiID)
	req.Header.Set("X-VO-Api-Key", apiKey)
	client := &http.Client{}

	resp, err := client.Do(req)
	if err != nil {
		logger.Log(fmt.Sprintf("Failed to make request to VictorOps. ERROR: %s", err))
		return nil, err
	}
	if resp.StatusCode != 200 {
		return nil, fmt.Errorf("%s", resp.Status)
	}
	defer func() { _ = resp.Body.Close() }()

	response := &OnCallResponse{}
	if err := json.NewDecoder(resp.Body).Decode(response); err != nil {
		logger.Log(fmt.Sprintf("Failed to decode JSON response. ERROR: %s", err))
		return nil, err
	}

	teams := parseTeams(response)
	return teams, nil
}

func parseTeams(input *OnCallResponse) []OnCallTeam {
	var teamResults []OnCallTeam

	for _, data := range input.TeamsOnCall {
		var team OnCallTeam

View on GitHub (pinned to bb838c1ccb)

Solutions

  1. Check the HTTP status in the message and fix accordingly: 401/403 → verify the apikey config value against the VictorOps REST API key
  2. Verify the organization slug / routing key in the widget config matches your VictorOps account
  3. Hit the API endpoint manually with curl using the same key to confirm the credential works
  4. If 5xx, retry later or check the Splunk On-Call status page for incidents

Example fix

# before
apikey: "old-revoked-key"
# after
apikey: "current-key-from-victorops-ui"
Defensive patterns

Strategy: retry

Validate before calling

// preflight: verify the key works before relying on the widget
curl -s -o /dev/null -w "%{http_code}" -H "X-VO-Api-Key: $KEY" https://alert.victorops.com/api/public/v1/team

Try / catch

data, err := w.Fetch()
if err != nil {
    if strings.Contains(err.Error(), "401") || strings.Contains(err.Error(), "403") {
        log.Printf("victorops auth failure, check apikey: %v", err)
    } else if s := err.Error(); strings.Contains(s, "5") { // 5xx
        time.Sleep(retryBackoff)
        data, err = w.Fetch()
    }
}

Prevention

When it happens

Trigger: Fetch calls victorOpsRequest, client.Do succeeds, but the VictorOps API responds with a status other than 200: 401/403 for a bad or missing API key, 404 for a bad slug/routing key, 5xx for service problems.

Common situations: Expired or wrong apikey in config, org slug typo'd, VictorOps outage or rate limiting, account migrated to Splunk On-Call with revoked keys.

Related errors


AI-assisted analysis of wtfutil/wtf@bb838c1ccb (2026-09-03). Data as JSON: /api/errors/ff8ffc762f749878. Report an issue: GitHub.