What causes "http get failed at %s" in Netflix/chaosmonkey?

Verify the Spinnaker API endpoint is reachable: curl the URL printed in the error from the machine running the code.

http get failed at %s

Error message

http get failed at %s

What it means

GetIssueIDs is not involved; this error is produced by Spinnaker.GetInstanceIDs when the HTTP GET to the Spinnaker API endpoint for the active ASG of a cluster fails at the transport level. The library wraps the underlying net/http error with the requested URL so the caller knows which endpoint was unreachable. It means the request never completed — DNS, connection, TLS, timeout, or request-construction failure — not that the server returned a bad status.

Source

Thrown at spinnaker/spinnaker.go:259

	for _, appName := range appNames {
		app, err := s.GetApp(appName)
		if err != nil {
			// If we have a problem with one app, we go to the next one
			log.Printf("WARNING: GetApp failed for %s: %v", appName, err)
			continue
		}

		c <- app
	}
}

// GetInstanceIDs gets the instance ids for a cluster
func (s Spinnaker) GetInstanceIDs(app string, account D.AccountName, cloudProvider string, region D.RegionName, cluster D.ClusterName) (D.ASGName, []D.InstanceID, error) {
	url := s.activeASGURL(app, string(account), string(cluster), cloudProvider, string(region))

	resp, err := s.client.Get(url)
	if err != nil {
		return "", nil, errors.Wrapf(err, "http get failed at %s", url)
	}

	defer func() {
		if cerr := resp.Body.Close(); cerr != nil && err == nil {
			err = errors.Wrapf(err, "body close failed at %s", url)
		}
	}()

	if resp.StatusCode != http.StatusOK {
		return "", nil, errors.Errorf("unexpected response code (%d) from %s", resp.StatusCode, url)
	}

	body, err := ioutil.ReadAll(resp.Body)
	if err != nil {
		return "", nil, errors.Wrap(err, fmt.Sprintf("body read failed at %s", url))
	}

	var data struct {

View on GitHub (pinned to eaa28fb761)

Solutions

Verify the Spinnaker API endpoint is reachable: curl the URL printed in the error from the machine running the code.
Check the client configuration (base URL, port, TLS) used to construct s.client and correct it.
Check DNS resolution and network/firewall rules to the Spinnaker gate host.
Increase the HTTP client timeout if requests are timing out under load.
Retry the call; transient network blips are a common cause.

Example fix

// before
client := &http.Client{}
sp := Spinnaker{client: client, ...}
// after
client := &http.Client{Timeout: 30 * time.Second}
sp := Spinnaker{client: client, ...} // also verify base URL points at the gate host

Defensive patterns

Strategy: retry

Validate before calling

// before calling, check reachability
u, err := url.Parse(baseURL)
if err != nil || u.Scheme == "" || u.Host == "" {
    return fmt.Errorf("invalid spinnaker base URL: %v", err)
}
conn, err := net.DialTimeout("tcp", u.Host, 5*time.Second)
if err != nil {
    return fmt.Errorf("spinnaker endpoint unreachable: %v", err)
}
conn.Close()

Try / catch

// Go: check and retry with backoff
var asg D.ASGName
var ids []D.InstanceID
for i := 0; i < 3; i++ {
    asg, ids, err = sp.GetInstanceIDs(app, acct, cp, region, cluster)
    if err == nil {
        break
    }
    var netErr net.Error
    if errors.As(err, &netErr) {
        time.Sleep(time.Duration(1<<i) * time.Second)
        continue
    }
    return err // non-retryable
}
if err != nil {
    log.Printf("GetInstanceIDs failed after retries: %+v", err)
}

Prevention

Configure the HTTP client with a sane timeout
Health-check the Spinnaker gate before workload starts
Validate base URL and credentials in config at startup
Log the full error chain (%+v with pkg/errors) to see the URL
Monitor gate availability

When it happens

Trigger: Calling GetInstanceIDs when the Spinnaker gate endpoint configured in the client is down, unreachable, the URL built by activeASGURL is malformed, DNS fails, TLS handshake fails, or the request times out or is cancelled before a response arrives.

Common situations: Spinnaker gate is not running or the configured host/port is wrong; network partition or firewall blocking the endpoint; DNS misconfiguration in cluster; expired TLS certificates; client-side timeouts during slow responses; typo in app/account/cluster inputs producing an invalid URL.

Understand the failure class

Background: 'Something went wrong' / 'Request failed (500)' / 'HTTP error! status: 404' — what failed HTTP requests actually mean and how to find the real cause — this error's family across 28 libraries.

Related errors

AI-assisted analysis of Netflix/chaosmonkey@eaa28fb761 (2026-09-03). Data as JSON: /api/errors/3ddb133c843f28d1. Report an issue: GitHub.