JuliusBrussee/caveman · error · ErrBodyTooLarge

redact: payload exceeds redaction size limit

Error message

redact: payload exceeds redaction size limit

What it means

Payload refuses bodies larger than MaxPayloadBytes, returning ErrBodyTooLarge (this message) wrapped with the actual size and the limit. Redaction operates on the full opaque body in memory, so oversized payloads are rejected instead of being partially redacted or exhausting memory.

Solutions

1. Check len(body) against redact.MaxPayloadBytes before calling Payload and reject/stream the request earlier
2. Split or truncate oversized payloads upstream (chunk the upload) so each piece fits the limit
3. Raise MaxPayloadBytes deliberately if the workload legitimately needs larger bodies, accounting for memory
4. Return a 413 Payload Too Large to the client with the documented limit

Example fix

// before
redacted, report, err := redact.Payload(body, rules) // body is 10MB, limit exceeded
// after
if len(body) > redact.MaxPayloadBytes {
    return http.StatusRequestEntityTooLarge
}
redacted, report, err := redact.Payload(body, rules)

if len(body) > redact.MaxPayloadBytes {
    return nil, fmt.Errorf("body %d bytes exceeds redaction limit %d", len(body), redact.MaxPayloadBytes)
}

redacted, report, err := redact.Payload(body, rules)
if err != nil {
    if errors.Is(err, redact.ErrBodyTooLarge) {
        return http.StatusRequestEntityTooLarge
    }
    return err
}

When it happens

Trigger: Calling Payload(body, rules) where len(body) > MaxPayloadBytes — e.g. uploading multi-megabyte JSON/log bodies through a redaction middleware.

Common situations: Clients POSTing large exports or log bundles to an endpoint protected by the redact middleware; a limit configured for small API payloads suddenly receiving file-like uploads; limit lowered in a new release making previously-fine payloads too big.

Understand the failure class

Background: payload too large / request exceeds maximum size: why libraries cap bytes and how to fix oversize payloads — this error's family across 50 libraries.

Related errors

• : bytes (limit )
• cave_sandbox_input_limit
• clickhouse response exceeds
• githubapp: repository proof has invalid type, encoding, or…
• hook payload too large

AI-assisted analysis of JuliusBrussee/caveman@3ee70a1026 (2026-09-20). Data as JSON: /api/errors/d496156a0dfe3430. Report an issue: GitHub.

Appendix: source

Thrown at shared/platform/redact/payload.go:68

// than being silently ignored, because ignoring a rule an operator configured
// is exactly how unredacted data leaks.
const (
	RuleTypeRegex    = "regex"
	RuleTypeJSONPath = "json_path"
	RuleTypeHeader   = "header"
	RuleTypeBuiltin  = "builtin"
)

// Origin values distinguish the built-in floor from operator-supplied rules in
// the report.
const (
	OriginBuiltin = "builtin"
	OriginOrg     = "org"
)

var (
	// ErrBodyTooLarge means the body exceeded MaxPayloadBytes.
	ErrBodyTooLarge = errors.New("redact: payload exceeds redaction size limit")
	// ErrInvalidRule means a rule could not be compiled or was structurally
	// unusable (empty name/pattern, or a pattern that matches the empty
	// string and would therefore rewrite the entire body).
	ErrInvalidRule = errors.New("redact: invalid redaction rule")
	// ErrRuleUnsupported means the rule's type is not implementable against an
	// opaque body by this pass.
	ErrRuleUnsupported = errors.New("redact: unsupported redaction rule type")
)

// Rule is one enabled row of the redaction_rules table, minus the columns that
// are the caller's business. Field names and JSON tags are pinned to that
// table's columns so the two cannot drift.
//
// Deliberately absent:
//   - id / organization_id / project_id — tenancy is the caller's concern.
//   - enabled — filtering disabled rows is the caller's concern; a rule that
//     reaches Payload is a rule that runs.
//

View on GitHub (pinned to 3ee70a1026)