What causes "failed to read request body: %w" in temporalio/temporal?

Retry the request — this is typically a transient network/stream error, not a config problem

failed to read request body

Error message

failed to read request body: %w

What it means

awsSigningTransport.RoundTrip must read the full request body into memory to compute the SHA-256 payload hash required by AWS SigV4 signing. If io.ReadAll on req.Body fails, the request is aborted with this wrapped error and never sent to Elasticsearch.

Solutions

Retry the request — this is typically a transient network/stream error, not a config problem
Check for middleware that reads req.Body without resetting it via io.NopCloser(bytes.NewReader(...)) before the AWS signing transport
Inspect the wrapped error for proxy/TLS connection resets and fix the underlying network path
Reduce request payload size if timeouts on large bodies are implicated

Example fix

// before: middleware consumed body and did not rewind
body, _ := io.ReadAll(req.Body)
req.Body = nil
// after: always restore the body
body, _ := io.ReadAll(req.Body)
req.Body = io.NopCloser(bytes.NewReader(body))

Defensive patterns

Strategy: retry

Validate before calling

// ensure any middleware has not consumed the body: it must be non-nil and unread
if req.Body != nil {
    if req.GetBody == nil && req.ContentLength > 0 {
        // body may not be rewindable; avoid chaining readers before the signing transport
    }
}

Try / catch

resp, err := client.Do(req)
if err != nil && strings.Contains(err.Error(), "failed to read request body") {
    // transient I/O — rebuild request and retry
    ...
}

Prevention

Never read req.Body in middleware without restoring it via io.NopCloser(bytes.NewReader(b))
Place the AWS signing transport outermost or ensure earlier roundtrippers don't consume streams
Keep visibility request payloads within sane size limits to reduce mid-stream faults
Monitor proxy/TLS connection resets in the visibility network path

When it happens

Trigger: Any visibility request whose req.Body stream errors during read: connection reset to an intermediate proxy mid-write, body already consumed/closed by an earlier middleware, or an I/O fault in a custom wrapped transport below the signer.

Common situations: Pipelined/proxied HTTP where an upstream connection breaks while streaming a large bulk-search query; a middleware chain that read and did not rewind the body before the signing transport; transient network faults in busy visibility clusters with large requests.

Related errors

AI-assisted analysis of temporalio/temporal@bde624efd1 (2026-09-01). Data as JSON: /api/errors/e83d0024ac4d4a82. Report an issue: GitHub.

Appendix: source

Thrown at common/persistence/visibility/store/elasticsearch/client/aws.go:38

type awsSigningTransport struct {
	creds   aws.CredentialsProvider
	signer  *v4signer.Signer
	region  string
	service string
	wrapped http.RoundTripper
}

func (t *awsSigningTransport) RoundTrip(req *http.Request) (*http.Response, error) {
	creds, err := t.creds.Retrieve(req.Context())
	if err != nil {
		return nil, fmt.Errorf("failed to retrieve AWS credentials: %w", err)
	}

	var bodyBytes []byte
	if req.Body != nil {
		bodyBytes, err = io.ReadAll(req.Body)
		if err != nil {
			return nil, fmt.Errorf("failed to read request body: %w", err)
		}
		req.Body = io.NopCloser(bytes.NewReader(bodyBytes))
	}

	hash := fmt.Sprintf("%x", sha256.Sum256(bodyBytes))
	err = t.signer.SignHTTP(req.Context(), creds, req, hash, t.service, t.region, time.Now())
	if err != nil {
		return nil, fmt.Errorf("failed to sign request: %w", err)
	}

	if bodyBytes != nil {
		// set the request body just in case the signer consumes the body
		req.Body = io.NopCloser(bytes.NewReader(bodyBytes))
	}

	return t.wrapped.RoundTrip(req)
}

View on GitHub (pinned to bde624efd1)