What causes "kms: read %s response: %w" in JuliusBrussee/caveman?

Retry the operation with exponential backoff — this is typically transient.

kms: read response

Error message

kms: read %s response: %w

What it means

The KMS response was received with HTTP 200, but reading the response body via io.ReadAll failed (connection reset mid-body, context cancellation, unexpected EOF). The client wraps the read error with the operation name. The request succeeded at the HTTP level but the full body could not be consumed.

Solutions

Retry the operation with exponential backoff — this is typically transient.
Check intermediary proxies/load balancers for response idle/read timeout settings.
Inspect the wrapped cause for context cancellation; ensure the caller's ctx isn't being cancelled prematurely.
Increase keep-alive robustness (Transport settings) or disable connection reuse if an intermediary is killing idle connections.
Check server-side logs for the request to confirm the server finished writing the response.

Example fix

// before
out, err := kms.Decrypt(ctx, ct) // kms: read Decrypt response: unexpected EOF
// after
var out DecryptOutput
err := retry.Do(3, backoff, func() error {
    var e error
    out, e = kms.Decrypt(ctx, ct)
    return e
})

Defensive patterns

Strategy: retry

Try / catch

if err != nil && strings.HasPrefix(err.Error(), "kms: read ") {
    var netErr net.Error
    if errors.As(err, &netErr) && netErr.Timeout() {
        // retry with backoff
    }
    return fmt.Errorf("reading KMS response: %w", err)
}

Prevention

Wrap KMS calls in bounded retries with exponential backoff.
Check proxy/LB idle-timeout settings against worst-case response times.
Avoid cancelling the request context prematurely on large payloads.
Monitor for connection resets between client and KMS.

When it happens

Trigger: Encrypt or Decrypt calls where the server closes the connection while the body is streaming, the request context is cancelled mid-read, LB idle timeout truncating the response, or transient socket errors during body transfer.

Common situations: Large ciphertext/large responses over flaky links, reverse proxies with short read timeouts, pod/network interruption in Kubernetes during the response, or keep-alive connections killed by an intermediary.

Related errors

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/38c0471d8b3c8ee1. Report an issue: GitHub.

Appendix: source

Thrown at shared/platform/kms/kms.go:371

	req, err := http.NewRequestWithContext(ctx, http.MethodPost, endpoint, bytes.NewReader(body))
	if err != nil {
		return fmt.Errorf("kms: create %s request: %w", operation, err)
	}
	req.Header.Set("content-type", "application/json")
	req.Header.Set("accept", "application/json")
	req.Header.Set("x-auth-token", c.token)
	resp, err := c.httpClient.Do(req)
	if err != nil {
		return fmt.Errorf("kms: %s request failed: %w", operation, err)
	}
	defer resp.Body.Close()
	if resp.StatusCode != http.StatusOK {
		_, _ = io.Copy(io.Discard, io.LimitReader(resp.Body, 32<<10))
		return fmt.Errorf("kms: %s returned HTTP %d", operation, resp.StatusCode)
	}
	data, err := io.ReadAll(io.LimitReader(resp.Body, maxResponseBytes+1))
	if err != nil {
		return fmt.Errorf("kms: read %s response: %w", operation, err)
	}
	if len(data) > maxResponseBytes {
		return fmt.Errorf("kms: %s response exceeds limit", operation)
	}
	if err := json.Unmarshal(data, output); err != nil {
		return fmt.Errorf("kms: decode %s response: %w", operation, err)
	}
	return nil
}

View on GitHub (pinned to 3ee70a1026)