crowdsecurity/crowdsec · error

body size exceeds max body size: %d

Error message

body size exceeds max body size: %d

What it means

rejectBody converts an *http.MaxBytesError (returned when the request body exceeds the configured MaxBodySize) into this error while responding to the client with HTTP 413. It is the datasource's way of aborting ingestion of an oversized payload.

Source

Thrown at pkg/acquisition/modules/http/run.go:53

			return errors.New("invalid basic auth")
		}
	}

	if hc.AuthType == "headers" {
		for key, value := range hc.Headers {
			if r.Header.Get(key) != value {
				return errors.New("invalid headers")
			}
		}
	}

	return nil
}

func rejectBody(w http.ResponseWriter, err error) error {
	if maxBytesErr, ok := errors.AsType[*http.MaxBytesError](err); ok {
		w.WriteHeader(http.StatusRequestEntityTooLarge)
		return fmt.Errorf("body size exceeds max body size: %d", maxBytesErr.Limit)
	}

	w.WriteHeader(http.StatusBadRequest)

	return fmt.Errorf("failed to read body: %w", err)
}

func (s *Source) processRequest(w http.ResponseWriter, r *http.Request, hc *Configuration, out chan pipeline.Event) error {
	// Shortcut for clients announcing an oversized body, so we don't read it at all.
	if hc.MaxBodySize != nil && r.ContentLength > *hc.MaxBodySize {
		w.WriteHeader(http.StatusRequestEntityTooLarge)
		return fmt.Errorf("body size exceeds max body size: %d > %d", r.ContentLength, *hc.MaxBodySize)
	}

	srcHost, _, err := net.SplitHostPort(r.RemoteAddr)
	if err != nil {
		return err
	}

View on GitHub (pinned to 909b515798)

Solutions

1. Raise max_body_size in the http datasource configuration to accommodate your largest expected payload.
2. Split or batch the client's payload into smaller requests below the limit.
3. Compress the payload (gzip) before sending if the client supports it.
4. Verify with the trace log which limit was hit — the message contains the exact byte limit.

Example fix

# before
max_body_size: 1M
# after
max_body_size: 10M

if len(payload) > maxBodySize { return errors.New("payload too large for datasource max_body_size") }

if resp.StatusCode == http.StatusRequestEntityTooLarge { // shrink batch and retry }

When it happens

Trigger: A client POSTs to the http datasource's endpoint and the body read hits the http.MaxBytesReader limit set from hc.MaxBodySize; processRequest calls rejectBody with the MaxBytesError.

Common situations: A log shipper or CI system posts a very large batch (big JSON array, large dump) while the datasource config sets a small max_body_size; default limits rejecting legitimate bulk uploads.

Understand the failure class

Background: payload too large / request exceeds maximum size: why libraries cap bytes and how to fix oversize payloads — this error's family across 50 libraries.

Related errors

• body size exceeds max body size: %d > %d
• path must start with /
• chunk_size must be positive
• invalid HTTP status code
• missing basic auth

AI-assisted analysis of crowdsecurity/crowdsec@909b515798 (2026-09-06). Data as JSON: /api/errors/2f60945b1862e629. Report an issue: GitHub.