JuliusBrussee/caveman · error

awscreds: read response

Error message

awscreds: read %s response: %w

What it means

doJSON wraps an error from reading the response body (io.ReadAll over a maxBody-bounded LimitReader) for a container or IMDS request. The connection opened and the status was received, but the body could not be fully read — typically the connection dropped mid-response.

Solutions

1. Retry the request — the metadata endpoints are idempotent GET/PUT and transient resets usually clear.
2. Check metadata service health on the host (curl -v the endpoint) to reproduce the mid-body drop.
3. Disable suspicious proxies/middleboxes between the process and the metadata address.
4. Inspect the wrapped cause (unexpected EOF, connection reset) to target the network fix.

Example fix

// before
body, err := doJSON(client, req, "imds credentials")
if err != nil { return nil, err } // single attempt
// after
body, err := retry(3, func() ([]byte, error) {
    return doJSON(client, req, "imds credentials")
})

body, err := doJSON(client, req, what)
if err != nil {
    if errors.Is(err, io.ErrUnexpectedEOF) || strings.Contains(err.Error(), "connection reset") {
        // bounded retry with backoff; metadata endpoints are idempotent
    }
}

When it happens

Trigger: The metadata server closes or resets the connection while the body is being read; network interruption mid-response; a proxy terminating the connection early; the peer sending a body larger than the reader tolerates in a way that breaks the connection.

Common situations: Flaky metadata service under instance overload; NAT or security appliance killing idle keep-alive connections mid-transfer; buggy local metadata simulator that half-closes responses.

Related errors

• awscreds: read sts response
• awscreds: request failed
• awscreds: AWS_EC2_METADATA_SERVICE_ENDPOINT is not a valid…
• awscreds: build imds token request
• awscreds: build request

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/e2d7ebfd7310b5d0. Report an issue: GitHub.

Appendix: source

Thrown at shared/platform/awscreds/awscreds.go:610

	req, err := http.NewRequestWithContext(ctx, http.MethodGet, endpoint, nil)
	if err != nil {
		return nil, fmt.Errorf("awscreds: build %s request: %w", what, err)
	}
	req.Header.Set("X-aws-ec2-metadata-token", token)
	return p.doJSON(p.link, req, what)
}

// doJSON performs one attempt and returns the bounded body. A non-2xx response
// is reported by status only: a metadata body holds credential material.
func (p *Provider) doJSON(client *http.Client, req *http.Request, what string) ([]byte, error) {
	resp, err := client.Do(req)
	if err != nil {
		return nil, fmt.Errorf("awscreds: %s request failed: %w", what, err)
	}
	defer resp.Body.Close()
	body, err := io.ReadAll(io.LimitReader(resp.Body, maxBody))
	if err != nil {
		return nil, fmt.Errorf("awscreds: read %s response: %w", what, err)
	}
	if resp.StatusCode < 200 || resp.StatusCode > 299 {
		return nil, fmt.Errorf("awscreds: %s: http %d", what, resp.StatusCode)
	}
	return body, nil
}

func credentialsFromJSON(body []byte, source string) (*result, error) {
	var parsed credentialJSON
	if err := json.Unmarshal(body, &parsed); err != nil {
		return nil, fmt.Errorf("awscreds: %s returned an unparseable response", source)
	}
	if parsed.Code != "" && !strings.EqualFold(parsed.Code, "Success") {
		return nil, fmt.Errorf("awscreds: %s returned code %q", source, parsed.Code)
	}
	expires, err := parseExpiry(parsed.Expiration)
	if err != nil {
		return nil, fmt.Errorf("awscreds: %s credential expiry: %w", source, err)

View on GitHub (pinned to 3ee70a1026)