Netflix/chaosmonkey · error

body read failed at %s

Error message

body read failed at %s

What it means

Spinnaker.GetInstanceIDs throws this when ioutil.ReadAll fails while reading the response body from a 200-OK response. The library wraps the read error with the URL. This usually means the connection was cut off mid-body — the response was truncated or the stream errored before EOF.

Source

Thrown at spinnaker/spinnaker.go:274

	resp, err := s.client.Get(url)
	if err != nil {
		return "", nil, errors.Wrapf(err, "http get failed at %s", url)
	}

	defer func() {
		if cerr := resp.Body.Close(); cerr != nil && err == nil {
			err = errors.Wrapf(err, "body close failed at %s", url)
		}
	}()

	if resp.StatusCode != http.StatusOK {
		return "", nil, errors.Errorf("unexpected response code (%d) from %s", resp.StatusCode, url)
	}

	body, err := ioutil.ReadAll(resp.Body)
	if err != nil {
		return "", nil, errors.Wrap(err, fmt.Sprintf("body read failed at %s", url))
	}

	var data struct {
		Name      string
		Instances []struct{ Name string }
	}

	err = json.Unmarshal(body, &data)
	if err != nil {
		return "", nil, errors.Wrapf(err, "failed to parse json at %s", url)
	}

	asg := D.ASGName(data.Name)
	instances := make([]D.InstanceID, len(data.Instances))
	for i, instance := range data.Instances {
		instances[i] = D.InstanceID(instance.Name)
	}

View on GitHub (pinned to eaa28fb761)

Solutions

1. Retry the request; transient connection resets are the usual cause.
2. Check proxy/LB idle and response timeouts and raise them for slow endpoints.
3. Ensure the HTTP client timeout is long enough to receive the full body.
4. Check network stability between the client and Spinnaker gate.
5. If bodies are consistently large, paginate or reduce the data requested.

Example fix

// before
client := &http.Client{}
// after
client := &http.Client{Timeout: 60 * time.Second} // allow full body transfer
// and/or retry:
for i := 0; i < 3; i++ {
    asg, ids, err := sp.GetInstanceIDs(app, acct, cp, region, cluster)
    if err == nil { break }
}

// confirm network path is stable before batch operations
if err := pingEndpoint(baseURL); err != nil {
    return fmt.Errorf("spinnaker unreachable, skipping batch: %w", err)
}

// Go: retry with backoff on read errors
for attempt := 0; attempt < 3; attempt++ {
    asg, ids, err := sp.GetInstanceIDs(app, acct, cp, region, cluster)
    if err == nil {
        return asg, ids, nil
    }
    if strings.Contains(err.Error(), "body read failed") {
        time.Sleep(time.Duration(1<<attempt) * 500 * time.Millisecond)
        continue
    }
    return asg, ids, err
}
return "", nil, errors.New("body read failed after retries")

When it happens

Trigger: Calling GetInstanceIDs when the TCP connection is reset or times out while the body is being transferred, a proxy or load balancer terminates the connection early, or the server aborts the response mid-stream.

Common situations: Large cluster listings over flaky networks; aggressive LB idle timeouts killing slow responses; proxy (e.g. corporate proxy) dropping chunked responses; Spinnaker gateway restarts mid-request.

Related errors

• body read failed at %s
• could not retrieve list of apps from spinnaker url %s: %v
• failed to read body when retrieving spinnaker app names from
• failed to retrieve server groups url (%s): %v
• http get failed at %s

AI-assisted analysis of Netflix/chaosmonkey@eaa28fb761 (2026-09-03). Data as JSON: /api/errors/9d0df24a5f58fd2c. Report an issue: GitHub.