{"record":{"id":"593312f3a7fd4a90","repo":"paperclipai/paperclip","slug":"artifact-exceeds-size-limit","errorCode":null,"errorMessage":"Artifact exceeds size limit.","messagePattern":"Artifact exceeds size limit\\.","errorType":"console","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"scripts/cloud-migrator-artifacts.mjs","lineNumber":146,"sourceCode":"    exec(\"npm\", [\"ci\", \"--ignore-scripts\", \"--no-audit\", \"--no-fund\", \"--update-notifier=false\", \"--cache\", path.join(scratch, \"empty-cache\"),\n      \"--registry=https://registry.npmjs.org\"], { cwd: scratch, stdio: \"inherit\", timeout: 180_000 });\n    for (const name of names) assertMetadata(JSON.parse(readFileSync(path.join(scratch, \"node_modules\", \"@paperclipai\", name, \"package.json\"), \"utf8\")), `@paperclipai/${name}`, sha);\n    exec(process.execPath, [\"--input-type=module\", \"--eval\", \"await import('@paperclipai/db'); await import('@paperclipai/shared');\"], { cwd: scratch, stdio: \"inherit\", timeout: 30_000 });\n  } finally { rmSync(scratch, { recursive: true, force: true }); }\n}\n\nasync function download(url, fetchImpl) {\n  const response = await fetchImpl(url, { redirect: \"error\", signal: AbortSignal.timeout(60_000) });\n  if (!response.ok) throw new Error(`Artifact download failed: HTTP ${response.status}`, { cause: { status: response.status } });\n  const reader = response.body.getReader();\n  const chunks = [];\n  let size = 0;\n  try {\n    while (true) {\n      const { done, value } = await reader.read();\n      if (done) break;\n      size += value.length;\n      if (size > maximumBytes) throw new Error(\"Artifact exceeds size limit.\");\n      chunks.push(value);\n    }\n  } finally { await reader.cancel(); }\n  return Buffer.concat(chunks);\n}\n\nexport async function verifyPublished(sha, fetchImpl = fetch, { verifyProvenance } = {}) {\n  versionFor(sha);\n  const bytes = await download(`${artifactBase}/${sha}/manifest.json`, fetchImpl);\n  const manifest = JSON.parse(bytes);\n  assertManifest(manifest, sha);\n  if (verifyProvenance) await verifyProvenance(bytes, sha);\n  await Promise.all(names.map(async (name) => {\n    const bytes = await download(manifest.packages[name].url, fetchImpl);\n    verifyBytes(bytes, manifest.packages[name]);\n    assertMetadata(tarManifest(bytes), `@paperclipai/${name}`, sha);\n  }));\n  const lock = await download(manifest.lockfile.url, fetchImpl);","sourceCodeStart":128,"sourceCodeEnd":164,"githubUrl":"https://github.com/paperclipai/paperclip/blob/3f1d897a7c018d76563a21c6e39c3c9b03933622/scripts/cloud-migrator-artifacts.mjs#L128-L164","documentation":"download() streams the response body and enforces a hard cap of 32 MiB (maximumBytes) accumulated before buffering the artifact. This error is thrown as soon as the running size exceeds the cap, protecting against a malicious or misconfigured endpoint returning an enormous payload. The reader is cancelled in the finally block.","triggerScenarios":"download() is called on a URL (manifest, tgz, or lockfile) whose streamed body exceeds 32 MiB — a wrong URL returning a huge file, a compromised/misbehaving CDN path, or a manifest pin pointing at the wrong blob.","commonSituations":"A typod or attacker-controlled URL serving arbitrary large content; the manifest's blob URL hijacked; fetching a directory listing or error page that is unexpectedly huge; fetching the wrong endpoint (e.g. an S3 bucket listing instead of a blob).","solutions":["Verify the URL matches the expected content-addressed form `${artifactBase}/blobs/<sha512hex>.<ext>` before fetching","Re-fetch the manifest from `${artifactBase}/<sha>/manifest.json` and re-run verifyPublished so pins and URLs come from the trusted manifest","If a legitimate artifact genuinely outgrew 32 MiB, raise maximumBytes deliberately in scripts/cloud-migrator-artifacts.mjs after confirming the blob's real size"],"exampleFix":"// before (wrong URL, unbounded response)\nconst bytes = await download(\"https://example.com/huge-dump.tgz\", fetch);\n// after (fetch the pinned content-addressed blob)\nconst manifest = JSON.parse(await download(`${artifactBase}/${sha}/manifest.json`, fetch));\nconst bytes = await download(manifest.packages.db.url, fetch); // asserted <= 32MiB by assertDescriptor","handlingStrategy":"try-catch","validationCode":"const head = await fetch(url, { method: \"HEAD\" });\nconst size = Number(head.headers.get(\"content-length\") ?? 0);\nif (size > 32 * 1024 * 1024) throw new Error(`artifact too large: ${size} bytes`);","typeGuard":"const withinArtifactLimit = (res) =>\n  Number(res.headers?.get?.(\"content-length\") ?? 0) <= 32 * 1024 * 1024;","tryCatchPattern":"try {\n  const bytes = await download(url, fetch);\n} catch (err) {\n  if (err.message === \"Artifact exceeds size limit.\") {\n    // wrong or hostile URL; re-resolve from the trusted manifest pins\n    const manifest = JSON.parse(await download(`${artifactBase}/${sha}/manifest.json`, fetch));\n    url = manifest.packages.db.url;\n  } else throw err;\n}","preventionTips":["Fetch only content-addressed blob URLs taken from an already-verified manifest","Never pass unverified user-supplied URLs to download()","Check content-length with a HEAD request before streaming large payloads","Treat size-limit violations as a security signal, not just a capacity issue"],"tags":["network","download","size-limit","security"],"backgroundTag":"file-size-limit-exceeded","analyzedSha":"3f1d897a7c018d76563a21c6e39c3c9b03933622","analyzedAt":"2026-09-18T08:03:59.046Z","contentChangedAt":"2026-09-18T08:03:59.046Z","schemaVersion":2},"datasetVersion":"2026-09-22T06:17:15.046Z"}