crowdsecurity/crowdsec · error

body size exceeds max body size: %d > %d

Error message

body size exceeds max body size: %d > %d

What it means

processRequest pre-checks the declared Content-Length against the configured MaxBodySize before reading anything, returning 413 to the client with this error naming both the declared size and the limit.

Source

Thrown at pkg/acquisition/modules/http/run.go:65

	return nil
}

func rejectBody(w http.ResponseWriter, err error) error {
	if maxBytesErr, ok := errors.AsType[*http.MaxBytesError](err); ok {
		w.WriteHeader(http.StatusRequestEntityTooLarge)
		return fmt.Errorf("body size exceeds max body size: %d", maxBytesErr.Limit)
	}

	w.WriteHeader(http.StatusBadRequest)

	return fmt.Errorf("failed to read body: %w", err)
}

func (s *Source) processRequest(w http.ResponseWriter, r *http.Request, hc *Configuration, out chan pipeline.Event) error {
	// Shortcut for clients announcing an oversized body, so we don't read it at all.
	if hc.MaxBodySize != nil && r.ContentLength > *hc.MaxBodySize {
		w.WriteHeader(http.StatusRequestEntityTooLarge)
		return fmt.Errorf("body size exceeds max body size: %d > %d", r.ContentLength, *hc.MaxBodySize)
	}

	srcHost, _, err := net.SplitHostPort(r.RemoteAddr)
	if err != nil {
		return err
	}

	// Content-Length can be absent (chunked, HTTP/2) or a lie, so bound what we actually read.
	// This also caps gzip streams that consume input without producing output.
	if hc.MaxBodySize != nil {
		r.Body = http.MaxBytesReader(w, r.Body, *hc.MaxBodySize)
	}

	defer r.Body.Close()

	if s.logger.Logger.IsLevelEnabled(log.TraceLevel) {
		s.logger.Tracef("processing request from '%s' with method '%s' and path '%s'", r.RemoteAddr, r.Method, r.URL.Path)

View on GitHub (pinned to 909b515798)

Solutions

1. Increase max_body_size in the datasource configuration.
2. Have the client batch/chunk requests so each Content-Length is under the limit.
3. Configure the client to stream with chunked encoding (no Content-Length) only if the limit is meant to apply to actual bytes — note the MaxBytesReader still enforces it on read.
4. Drop explicit Content-Length mismatch by fixing the client's pre-computed length if it is wrong.

Example fix

// client before
body, _ := json.Marshal(allEvents)
// after (batching)
for _, batch := range chunk(allEvents, 1000) { post(json.Marshal(batch)) }

if r.ContentLength > maxBodySize { // do not send; split batch first }

if resp.StatusCode == http.StatusRequestEntityTooLarge { splitBatchAndRetry() }

When it happens

Trigger: Client sends a request with Content-Length greater than hc.MaxBodySize (both non-nil); the check runs before any body bytes are read.

Common situations: Bulk log exporters posting very large JSON payloads to a datasource configured with a conservative max_body_size; misconfigured clients sending entire log files in one request.

Understand the failure class

Background: payload too large / request exceeds maximum size: why libraries cap bytes and how to fix oversize payloads — this error's family across 50 libraries.

Related errors

• body size exceeds max body size: %d
• path must start with /
• chunk_size must be positive
• invalid HTTP status code
• missing basic auth

AI-assisted analysis of crowdsecurity/crowdsec@909b515798 (2026-09-06). Data as JSON: /api/errors/c1653dea48cfa430. Report an issue: GitHub.