wtfutil/wtf · error
%s
Error message
%s
What it means
victorOpsRequest treats any non-200 HTTP status from the VictorOps (Splunk On-Call) API as an error and returns fmt.Errorf("%s", resp.Status), so the message is the raw HTTP status line (e.g. '401 Unauthorized'). The library has no retry or body-based error extraction for this path — the status line is all you get.
Source
Thrown at modules/victorops/client.go:39
func victorOpsRequest(url string, apiID string, apiKey string) ([]OnCallTeam, error) {
req, err := http.NewRequest("GET", url, http.NoBody)
if err != nil {
logger.Log(fmt.Sprintf("Failed to initialize sessions to VictorOps. ERROR: %s", err))
return nil, err
}
req.Header.Set("X-VO-Api-Id", apiID)
req.Header.Set("X-VO-Api-Key", apiKey)
client := &http.Client{}
resp, err := client.Do(req)
if err != nil {
logger.Log(fmt.Sprintf("Failed to make request to VictorOps. ERROR: %s", err))
return nil, err
}
if resp.StatusCode != 200 {
return nil, fmt.Errorf("%s", resp.Status)
}
defer func() { _ = resp.Body.Close() }()
response := &OnCallResponse{}
if err := json.NewDecoder(resp.Body).Decode(response); err != nil {
logger.Log(fmt.Sprintf("Failed to decode JSON response. ERROR: %s", err))
return nil, err
}
teams := parseTeams(response)
return teams, nil
}
func parseTeams(input *OnCallResponse) []OnCallTeam {
var teamResults []OnCallTeam
for _, data := range input.TeamsOnCall {
var team OnCallTeamView on GitHub (pinned to bb838c1ccb)
Solutions
- Check the HTTP status in the message and fix accordingly: 401/403 → verify the apikey config value against the VictorOps REST API key
- Verify the organization slug / routing key in the widget config matches your VictorOps account
- Hit the API endpoint manually with curl using the same key to confirm the credential works
- If 5xx, retry later or check the Splunk On-Call status page for incidents
Example fix
# before apikey: "old-revoked-key" # after apikey: "current-key-from-victorops-ui"
Defensive patterns
Strategy: retry
Validate before calling
// preflight: verify the key works before relying on the widget
curl -s -o /dev/null -w "%{http_code}" -H "X-VO-Api-Key: $KEY" https://alert.victorops.com/api/public/v1/team Try / catch
data, err := w.Fetch()
if err != nil {
if strings.Contains(err.Error(), "401") || strings.Contains(err.Error(), "403") {
log.Printf("victorops auth failure, check apikey: %v", err)
} else if s := err.Error(); strings.Contains(s, "5") { // 5xx
time.Sleep(retryBackoff)
data, err = w.Fetch()
}
} Prevention
- Rotate and verify the VictorOps API key before it expires
- Monitor Splunk On-Call status for 5xx windows
- Alert on auth-status failures distinctly from transient 5xx
When it happens
Trigger: Fetch calls victorOpsRequest, client.Do succeeds, but the VictorOps API responds with a status other than 200: 401/403 for a bad or missing API key, 404 for a bad slug/routing key, 5xx for service problems.
Common situations: Expired or wrong apikey in config, org slug typo'd, VictorOps outage or rate limiting, account migrated to Splunk On-Call with revoked keys.
Related errors
AI-assisted analysis of wtfutil/wtf@bb838c1ccb (2026-09-03).
Data as JSON: /api/errors/ff8ffc762f749878.
Report an issue: GitHub.