{"record":{"id":"6578d7f56bcd2294","repo":"koala73/worldmonitor","slug":"feed-body-too-large","errorCode":null,"errorMessage":"Feed body too large","messagePattern":"Feed body too large","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"api/rss-proxy.js","lineNumber":175,"sourceCode":"\n        ITEM_CLOSE_PATTERN.lastIndex = scannedTo;\n        for (let m = ITEM_CLOSE_PATTERN.exec(result.data); m; m = ITEM_CLOSE_PATTERN.exec(result.data)) {\n          items += 1;\n          keepTo = m.index + m[0].length;\n          if (items >= MAX_FEED_ITEMS) break;\n        }\n        scannedTo = Math.max(keepTo, result.data.length - ITEM_CLOSE_SEAM);\n\n        if (items >= MAX_FEED_ITEMS || bytes > MAX_FEED_BYTES) {\n          bounded = true;\n          break;\n        }\n      }\n      result.data += decoder.decode();\n      if (bounded) {\n        // A single item wider than the byte cap leaves nothing well-formed to\n        // keep, so that stays a hard failure.\n        if (keepTo < 0) throw new Error('Feed body too large');\n        const kept = result.data.slice(0, keepTo);\n        result.data = kept + closeOpenElements(kept);\n        void reader.cancel().catch(() => {});\n      }\n      return result;\n    })()]);\n  } catch (error) {\n    controller.abort();\n    // Cancellation can itself stall. Request it, but never extend the deadline.\n    void reader?.cancel(error).catch(() => {});\n    throw error;\n  } finally {\n    clearTimeout(timer);\n    reader?.releaseLock();\n  }\n}\n\nclass RssProxyPolicyError extends Error {","sourceCodeStart":157,"sourceCodeEnd":193,"githubUrl":"https://github.com/koala73/worldmonitor/blob/e586b8b4b80f595aa7ece295eec10d76f2921240/api/rss-proxy.js#L157-L193","documentation":"The RSS proxy streams feed bodies through a bounded reader that tracks a byte cap. When a single RSS item is wider than the cap, there is no well-formed prefix worth keeping, so fetchRssResponse throws 'Feed body too large' as a hard failure instead of returning truncated, malformed XML. Smaller overflows are handled by slicing and closing open elements; this error is reserved for the unc salvageable case.","triggerScenarios":"Fetching an RSS feed whose response stream exceeds the configured byte cap while the accumulated data has not yet reached a valid item boundary (keepTo < 0), i.e. one enormous <item> larger than the entire allowed budget.","commonSituations":"A publisher emitting a feed with huge embedded content/CDATA blobs in one item; a misconfigured or malicious feed serving gigantic single entries; proxying a feed endpoint that ignores size limits.","solutions":["Increase the feed size cap in the rss-proxy configuration if legitimately large feeds must be supported.","Drop or exclude the offending feed source from the aggregation list.","Ask the feed publisher to split or trim oversized item content (e.g. full-article HTML in CDATA).","Pre-check the feed with a HEAD request for Content-Length and skip feeds exceeding the cap."],"exampleFix":null,"handlingStrategy":"try-catch","validationCode":"const head = await fetch(feedUrl, { method: 'HEAD' });\nconst len = Number(head.headers.get('content-length') || 0);\nif (len > MAX_FEED_BYTES) skipFeed(feedUrl);","typeGuard":null,"tryCatchPattern":"try { const feed = await fetchRssResponse(url); ... } catch (e) { if (e.message === 'Feed body too large') { markFeedOversized(url); return cachedFeed ?? null; } throw e; }","preventionTips":["Set a Content-Length pre-check with HEAD before streaming","Keep per-item content bounded at the publisher level","Cache the last good snapshot so an oversized feed degrades gracefully","Alert on feeds that repeatedly exceed the cap instead of failing silently"],"tags":["network","rss","payload-size","streaming"],"backgroundTag":"payload-too-large","analyzedSha":"e586b8b4b80f595aa7ece295eec10d76f2921240","analyzedAt":"2026-09-22T01:50:49.965Z","contentChangedAt":"2026-09-22T01:50:49.965Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}