What causes "read response body" in weaviate/weaviate?

Check the inner error ('unexpected EOF', 'connection reset by peer') and inspect inference container logs for OOM kills or restarts

weaviate/weaviate · error

read response body

Error message

read response body

What it means

Wrapping error from the ner-transformers client GetTokens when io.ReadAll fails to read the response body returned by the NER inference service, decorated as 'read response body'. This means the connection produced headers but the body stream broke mid-read — typically the inference container was killed/restarted (OOM or crash) or an intermediary severed the connection. The next statement also surfaces JSON parse problems with the HTTP status included, but this specific wrapper is the raw body-read I/O failure.

Source

Thrown at modules/ner-transformers/clients/ner.go:87

	if err != nil {
		return nil, errors.Wrapf(err, "marshal body")
	}

	req, err := http.NewRequestWithContext(ctx, "POST", n.url("/ner/"),
		bytes.NewReader(body))
	if err != nil {
		return nil, errors.Wrap(err, "create POST request")
	}

	res, err := n.httpClient.Do(req)
	if err != nil {
		return nil, errors.Wrap(err, "send POST request")
	}
	defer res.Body.Close()

	bodyBytes, err := io.ReadAll(res.Body)
	if err != nil {
		return nil, errors.Wrap(err, "read response body")
	}

	var resBody nerResponse
	if err := json.Unmarshal(bodyBytes, &resBody); err != nil {
		return nil, fmt.Errorf("failed to parse NER response (status %d): %w", res.StatusCode, err)
	}

	if res.StatusCode > 399 {
		return nil, errors.Errorf("fail with status %d: %s", res.StatusCode, resBody.Error)
	}

	out := make([]ent.TokenResult, len(resBody.Tokens))

	for i, elem := range resBody.Tokens {
		out[i].Certainty = elem.Certainty
		out[i].Distance = elem.Distance
		out[i].Entity = elem.Entity
		out[i].Word = elem.Word

View on GitHub (pinned to 75aa4b6d11)

Solutions

Check the inner error ('unexpected EOF', 'connection reset by peer') and inspect inference container logs for OOM kills or restarts
Shorten or chunk the input text so the NER model fits within container memory limits
Raise the inference container memory limit (e.g. docker memory / k8s resources.limits) and enable swap-free headroom
Rule out intermediary timeouts: increase LB/proxy response timeouts or bypass the proxy for internal module traffic
Retry the request; if transient resets persist, pin a healthy inference image version and monitor container restarts

Example fix

// before (docker-compose)
ner-transformers:
  image: semitechnologies/ner-transformers:latest
# after
ner-transformers:
  image: semitechnologies/ner-transformers:latest
  mem_limit: 4g  # prevent OOM kill mid-response

Defensive patterns

Strategy: retry

Validate before calling

// Keep inputs within safe size and confirm container headroom before large NER jobs
if len(text) > maxSafeChars {
	text = truncate(text, maxSafeChars)
}
// and: docker stats / kubectl top pod to verify inference container is not near its memory limit

Type guard

// Distinguish body-read breakage from parse errors so retries target the right cause
func isBodyReadError(err error) bool {
	msg := err.Error()
	return strings.Contains(msg, "read response body") ||
		strings.Contains(msg, "unexpected EOF") ||
		strings.Contains(msg, "connection reset")
}

Try / catch

tokens, err := nerClient.GetTokens(ctx, text)
if err != nil && strings.Contains(err.Error(), "read response body") {
	// transient stream break: retry with backoff; if persistent, check inference container for OOM kills
}

Prevention

Right-size inference container memory and truncate very long texts
Watch for OOM-killed / restarting inference pods (docker inspect, kubectl describe)
Avoid aggressive LB/proxy response timeouts on internal module traffic
Retry idempotent NER reads with capped exponential backoff
Pin a stable inference image version to rule out flaky builds

When it happens

Trigger: GetTokens received an HTTP response but reading its body errored: inference container OOM-killed mid-response (common with long texts on memory-limited transformers), abrupt connection reset, keep-alive connection closed by a proxy, or a very large response truncated by an intermediary.

Common situations: Long input texts pushing the NER model past container memory limits so the pod is killed while streaming; Docker/Kubernetes restarting the inference container; a load balancer with a short idle/response timeout; flaky network between Weaviate and the module sidecar.

Related errors

AI-assisted analysis of weaviate/weaviate@75aa4b6d11 (2026-09-04). Data as JSON: /api/errors/422fd67a4e325c38. Report an issue: GitHub.