What causes "http get failed at %s" in Netflix/chaosmonkey?

Check the wrapped cause for the transport error (connection refused vs DNS vs TLS)

http get failed at %s

Error message

http get failed at %s

What it means

Spinnaker.Get fetches an application's config from the Spinnaker gate API (with ?expand=false). If the HTTP client GET itself fails (connection error, DNS failure, TLS problem, timeout), the error is wrapped as 'http get failed at <url>'. No config can be retrieved, so app evaluation aborts.

Source

Thrown at spinnaker/config.go:32

package spinnaker

import (
	"io/ioutil"
	"net/http"

	"github.com/Netflix/chaosmonkey/v2"

	"github.com/pkg/errors"
)

// Get implements chaosmonkey.Getter.Get
func (s Spinnaker) Get(app string) (c *chaosmonkey.AppConfig, err error) {
	// avoid expanding the response to avoid unneeded load
	url := s.appURL(app) + "?expand=false"
	resp, err := s.client.Get(url)
	if err != nil {
		return nil, errors.Wrapf(err, "http get failed at %s", url)
	}

	defer func() {
		if cerr := resp.Body.Close(); cerr != nil && err == nil {
			err = errors.Wrapf(err, "body close failed at %s", url)
		}
	}()

	// should return a 200
	if resp.StatusCode != http.StatusOK {
		return nil, errors.Errorf("unexpected response code (%d) from %s", resp.StatusCode, url)
	}

	body, err := ioutil.ReadAll(resp.Body)
	if err != nil {
		return nil, errors.Wrapf(err, "body read failed at %s", url)
	}

View on GitHub (pinned to eaa28fb761)

Solutions

Check the wrapped cause for the transport error (connection refused vs DNS vs TLS)
Verify the Spinnaker gate URL/host configured for Spinnaker is correct and reachable (curl the URL)
Confirm Spinnaker gate service is healthy and running
Check network/firewall/DNS from the monkey's runtime environment
If transient, add retry with backoff around config fetches

Example fix

// before
resp, err := s.client.Get(url)
if err != nil {
    return nil, errors.Wrapf(err, "http get failed at %s", url)
}
// after (caller-side guard)
if err := cfgGateReachable(spinnakerURL); err != nil {
    log.Fatalf("spinnaker gate unreachable: %v", err)
}

Defensive patterns

Strategy: retry

Validate before calling

// reachability probe before running the monkey
resp, err := http.Get(gateBaseURL + "/health")
if err != nil {
    return fmt.Errorf("spinnaker gate unreachable at %s: %w", gateBaseURL, err)
}
resp.Body.Close()

Try / catch

cfg, err := getter.Get(app)
if err != nil {
    if strings.Contains(err.Error(), "http get failed") {
        log.Printf("transport error fetching %s config, retrying: %+v", app, err)
        // retry with backoff
    }
    return err
}

Prevention

Monitor Spinnaker gate health from the monkey's network segment
Configure the gate URL via a single, validated config value
Set client timeouts and retry with exponential backoff
Check TLS cert expiry for the gate endpoint proactively
Verify DNS resolution in the monkey's runtime environment

When it happens

Trigger: s.client.Get(url) returns a transport-level error: Spinnaker gate unreachable, DNS resolution failure, TLS handshake error, connection refused/reset, or request timeout.

Common situations: Wrong gate URL configured; Spinnaker down or restarting; network/firewall rules blocking the monkey's egress; expired TLS certificates; DNS issues in the cluster where the monkey runs.

Understand the failure class

Background: 'Something went wrong' / 'Request failed (500)' / 'HTTP error! status: 404' — what failed HTTP requests actually mean and how to find the real cause — this error's family across 28 libraries.

Related errors

AI-assisted analysis of Netflix/chaosmonkey@eaa28fb761 (2026-09-03). Data as JSON: /api/errors/ff761bcdf66165cb. Report an issue: GitHub.