{"record":{"id":"dad5dba16c6a2333","repo":"Hmbown/CodeWhale","slug":"response-exceeds-size-limit","errorCode":null,"errorMessage":"response exceeds size limit","messagePattern":"response exceeds size limit","errorType":"exception","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"web/scripts/facts-publish.mjs","lineNumber":486,"sourceCode":"  return text ? JSON.parse(text) : null;\n}\n\nexport async function readBoundedResponse(response, maxBytes = MAX_ENVELOPE_BYTES) {\n  const length = response.headers.get(\"content-length\");\n  if (length !== null && (!/^\\d+$/.test(length) || Number(length) > maxBytes)) {\n    await response.body?.cancel();\n    throw new Error(\"response exceeds size limit or has invalid length\");\n  }\n  if (!response.body) return \"\";\n  const reader = response.body.getReader();\n  const chunks = [];\n  let size = 0;\n  try {\n    while (true) {\n      const { value, done } = await reader.read();\n      if (done) break;\n      size += value.byteLength;\n      if (size > maxBytes) throw new Error(\"response exceeds size limit\");\n      chunks.push(value);\n    }\n  } catch (error) { try { await reader.cancel(); } catch { /* Keep rejection. */ } throw error; }\n  finally { reader.releaseLock(); }\n  return new TextDecoder(\"utf-8\", { fatal: true }).decode(Buffer.concat(chunks, size));\n}\n\nfunction readJson(path) {\n  return JSON.parse(new TextDecoder(\"utf-8\", { fatal: true }).decode(readBoundedFile(path)));\n}\n\nfunction nowIso() {\n  return new Date().toISOString().replace(/\\.\\d{3}Z$/, \"Z\");\n}\n\nasync function main(argv) {\n  const { positional, flags } = parseArgs(argv);\n  const cmd = positional[0];","sourceCodeStart":468,"sourceCodeEnd":504,"githubUrl":"https://github.com/Hmbown/CodeWhale/blob/433685b2024e7bc4c99e1e2e326bcad39b4d9d65/web/scripts/facts-publish.mjs#L468-L504","documentation":"Second half of readBoundedResponse()'s bound: when the response is streamed (no usable Content-Length), it accumulates chunks and throws as soon as the running byte total exceeds maxBytes. This enforces the same size cap as the header check but for chunked responses, then cancels the reader.","triggerScenarios":"A chunked (Transfer-Encoding: chunked) PostgREST response whose accumulated body bytes exceed MAX_ENVELOPE_BYTES, typically a query returning far more rows than expected.","commonSituations":"Forgetting the select=id or slug filter so the whole facts_channel/facts_key table is downloaded; a misconfigured Supabase returning an error page as a large HTML body; publishing envelopes that have legitimately grown beyond the cap.","solutions":["Narrow the PostgREST query with filters and select so only required rows are returned","Compare the real response size (curl | wc -c) against MAX_ENVELOPE_BYTES","Raise MAX_ENVELOPE_BYTES if envelopes legitimately exceed the current limit","Check for Supabase error responses being returned with 200 and a large body"],"exampleFix":null,"handlingStrategy":"validation","validationCode":"const head = await fetch(url, { method: 'HEAD' });\nif (Number(head.headers.get('content-length') ?? 0) > MAX_BYTES) throw new Error('too large');","typeGuard":null,"tryCatchPattern":"try {\n  const data = await readBoundedResponse(res);\n} catch (e) {\n  if (e.message === 'response exceeds size limit') console.error('Streamed body grew past cap; fix query filters or raise the cap');\n  throw e;\n}","preventionTips":["Scope every PostgREST query with filters/select","Increase the byte cap only deliberately, with review","Monitor for 200 responses with HTML error bodies"],"tags":["http","size-limit","streaming"],"backgroundTag":"file-size-limit-exceeded","analyzedSha":"433685b2024e7bc4c99e1e2e326bcad39b4d9d65","analyzedAt":"2026-09-15T12:24:24.634Z","contentChangedAt":"2026-09-15T12:24:24.634Z","schemaVersion":2},"datasetVersion":"2026-09-23T02:17:17.105Z"}