{"record":{"id":"f12bb1f739aa63a1","repo":"JuliusBrussee/caveman","slug":"w-d-bytes-limit-d","errorCode":null,"errorMessage":"%w: %d bytes (limit %d)","messagePattern":"%w: (.+?) bytes \\(limit (.+?)\\)","errorType":"validation","errorClass":"ErrBodyTooLarge","httpStatus":null,"severity":"error","filePath":"shared/platform/redact/payload.go","lineNumber":476,"sourceCode":"//\n// rules are the organization's enabled redaction_rules rows. They are additive\n// only — the built-in floor runs first and cannot be disabled, shadowed, or\n// reordered by anything a tenant configures.\n//\n// The returned slice ALIASES body when no rule fired; body is never mutated\n// (TestPayloadDoesNotMutateInput), so this is safe to read, but a caller that\n// intends to mutate the result in place must copy it first.\n//\n// Payload does not judge how BROAD a rule is. A pattern like \".+\" compiles,\n// matches, and collapses the whole body into one placeholder — a self-inflicted\n// loss of corpus value, not a leak, and the only pattern shapes rejected here\n// are the ones that are unusable rather than merely greedy. Rejecting breadth\n// belongs at rule-write time, where an operator is present to see the error;\n// there is no such surface yet (see the H1 report's concern on the missing\n// redaction_rules validator).\nfunc Payload(body []byte, rules []Rule) ([]byte, RedactionReport, error) {\n\tif len(body) > MaxPayloadBytes {\n\t\treturn nil, RedactionReport{}, fmt.Errorf(\"%w: %d bytes (limit %d)\", ErrBodyTooLarge, len(body), MaxPayloadBytes)\n\t}\n\torgRules, fingerprint, err := compileOrgRules(rules)\n\tif err != nil {\n\t\treturn nil, RedactionReport{}, err\n\t}\n\n\tsum := sha256.Sum256([]byte(builtinRuleSetFingerprint + \"\\x00\" + fingerprint))\n\treport := RedactionReport{\n\t\tRuleSetHash: hex.EncodeToString(sum[:16]),\n\t\tBytesIn:     len(body),\n\t}\n\n\tout := body\n\t// The prescreen is derived ONCE, then re-derived only after a rule whose\n\t// replacement can introduce a needle (see replIntroducesNeedle). Of the\n\t// built-ins only three can, and only when they actually fire.\n\tlowered := asciiLower(out)\n\t// Built-ins first: whatever an org rule does afterwards, it acts on a body","sourceCodeStart":458,"sourceCodeEnd":494,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/27d5a3981a347890211bb1bf2439e5c821a63bc9/shared/platform/redact/payload.go#L458-L494","documentation":"redact.Payload refuses to process a body larger than MaxPayloadBytes (8 MiB, payload.go:45) before doing any regex work. The error wraps ErrBodyTooLarge and reports the actual size and the limit. The cap exists so a hostile or oversized upload cannot turn the redaction pass (many regexes over the whole body) into a CPU/memory amplification vector.","triggerScenarios":"Calling redact.Payload(body, rules) with len(body) > 8<<20 — e.g. a captured HTTP request/response body, a log blob, or an artifact fed into the redaction pipeline wholesale.","commonSituations":"Redacting large API responses or binary-ish payloads that slipped past upstream size limits; a proxy/capture tool that buffers full bodies before redaction; increasing upload limits elsewhere in the app without updating this constant's expectations.","solutions":["Enforce an 8 MiB body limit upstream (request size middleware) so oversized inputs are rejected with 413 before reaching redaction.","If legitimate payloads exceed 8 MiB, redact in chunks that respect record boundaries (e.g. per log line) rather than raising the global cap — the cap is a DoS guard.","For huge text files, stream-filter with the same rule set instead of using Payload, accepting per-chunk rather than whole-body guarantees."],"exampleFix":"// before\nred, rep, err := redact.Payload(body, rules) // body is 50 MiB\n\n// after\nconst max = redact.MaxPayloadBytes\nif len(body) > max {\n    http.Error(w, fmt.Sprintf(\"body exceeds %d bytes\", max), http.StatusRequestEntityTooLarge)\n    return\n}\nred, rep, err := redact.Payload(body, rules)","handlingStrategy":"validation","validationCode":"if len(body) > redact.MaxPayloadBytes {\n    return fmt.Errorf(\"rejecting body of %d bytes (limit %d)\", len(body), redact.MaxPayloadBytes)\n}\nout, rep, err := redact.Payload(body, rules)","typeGuard":null,"tryCatchPattern":"out, rep, err := redact.Payload(body, rules)\nif errors.Is(err, redact.ErrBodyTooLarge) {\n    // reject upstream (413) or chunk the body along record boundaries; do not silently pass through unredacted\n}","preventionTips":["Enforce the same 8 MiB cap at ingestion (HTTP middleware) so rejection happens at the boundary.","Never 'recover' by redacting nothing — a too-large body must fail closed, not pass through.","Chunk large text along line/record boundaries instead of raising the global cap."],"tags":["redaction","limits","dos-hardening","size-limit"],"backgroundTag":null,"analyzedSha":"27d5a3981a347890211bb1bf2439e5c821a63bc9","analyzedAt":"2026-08-15T09:26:11.751Z","schemaVersion":2},"datasetVersion":"2026-08-15T22:17:37.221Z"}