{"record":{"id":"e9393fb9c323a7e8","repo":"JuliusBrussee/caveman","slug":"capacity","errorCode":"capacity","errorMessage":"capacity","messagePattern":"capacity","errorType":"error_code","errorClass":"MiddlewareError","httpStatus":null,"severity":"error","filePath":"packages/sdk/typescript/src/middleware/runtime.ts","lineNumber":327,"sourceCode":"    if (diagnostic) { try { this.options.onDiagnostic?.({ code, cacheContinuity: 'unavailable' }); } catch { /* sink cannot break requests */ } }\n    if (this.options.strict && diagnostic) throw new MiddlewareError(code);\n    return { status: 'bypassed', reason: code, replacements: [], plan: null, request: null, cacheContinuity: 'unavailable' };\n  }\n\n  private async http(path: string, body: string | undefined, timeoutMs: number, signal?: AbortSignal): Promise<unknown> {\n    const deadline = new AbortController();\n    const timer = setTimeout(() => deadline.abort(new MiddlewareError('deadline')),Math.max(1,Math.ceil(timeoutMs)));\n    try {\n      return await this.exchange(path,body,AbortSignal.any([this.lifetime.signal,deadline.signal,...(signal ? [signal] : [])]));\n    } finally { clearTimeout(timer); }\n  }\n\n  private async exchange(path: string, body: string | undefined, combined: AbortSignal): Promise<unknown> {\n    combined.throwIfAborted();\n    const headers: Record<string,string> = { 'Content-Type': 'application/json' };\n    if (this.options.token) headers['Authorization'] = `Bearer ${this.options.token}`;\n    const receipt = path === 'receipts';\n    if (receipt ? this.receiptFetchesPending >= 1 : this.fetchesPending >= 16) throw new MiddlewareError('capacity');\n    if (receipt) this.receiptFetchesPending++; else this.fetchesPending++;\n    const release = () => { if (receipt) this.receiptFetchesPending--; else this.fetchesPending--; };\n    const pending = Promise.resolve().then(() => this.fetcher(this.endpoint+PREFIX+path, { method: body === undefined ? 'GET' : 'POST', headers,\n      ...(body !== undefined ? { body } : {}), signal: combined, redirect: 'error' }));\n    // A custom transport may ignore abort. Bound its outstanding calls and\n    // dispose of any late body without letting it reopen a model dispatch.\n    void pending.then(response => { release(); if (combined.aborted) void response.body?.cancel().catch(() => {}); }, release);\n    const response = await untilAborted(pending,combined);\n    const reader = response.body?.getReader();\n    if (!reader) throw new MiddlewareError('invalid_plan');\n    let size = 0;\n    const chunks: Uint8Array[] = [];\n    try {\n      for (;;) {\n        const next = await untilAborted(reader.read(),combined);\n        if (next.done) break;\n        size += next.value.length;\n        if (size > 4 << 20) throw new MiddlewareError('payload_limit');","sourceCodeStart":309,"sourceCodeEnd":345,"githubUrl":"https://github.com/JuliusBrussee/caveman/blob/3ee70a102609e550bd2e68004bf5990a9341c851/packages/sdk/typescript/src/middleware/runtime.ts#L309-L345","documentation":"MiddlewareError('capacity') is thrown by the internal exchange() when the runtime's outbound HTTP concurrency limits are exhausted: at most 16 concurrent capability/optimize/receipt fetches, or at most 1 concurrent receipt fetch. This bounds outstanding calls so a slow or hung custom transport cannot accumulate unbounded work.","triggerScenarios":"More than 16 simultaneous optimize()/retrieve()/capabilities calls in flight (fetchesPending >= 16), or a second concurrent receipt fetch while receiptFetchesPending >= 1; especially when a custom fetch transport ignores abort and keeps calls outstanding.","commonSituations":"High fan-out code issuing many model requests concurrently through one shared runtime; a custom options.fetch implementation that hangs and never releases slots; a burst of optimize() calls during a slow runtime endpoint.","solutions":["Serialize or throttle optimize()/retrieve() calls, e.g. with a small promise pool (<=16)","Share one runtime instance but cap concurrent model dispatches","Fix or replace a custom fetch transport that ignores abort signals and leaks outstanding calls","Check for a prior stuck receipt fetch before issuing another"],"exampleFix":"// before\nawait Promise.all(requests.map(r => runtime.optimize(r))) // >16 concurrent\n// after\nimport pLimit from 'p-limit';\nconst limit = pLimit(8);\nawait Promise.all(requests.map(r => limit(() => runtime.optimize(r))))","handlingStrategy":"fallback","validationCode":null,"typeGuard":null,"tryCatchPattern":"try { return await runtime.optimize(req); } catch (e) { if (e instanceof MiddlewareError && e.code === 'capacity') { await delay(jitter()); return sendUnoptimized(req); } throw e; }","preventionTips":["Cap concurrent optimize/retrieve calls (<= 8-16) with a semaphore or p-limit","Never issue concurrent receipt fetches; await the previous one","Audit custom fetch transports so they honor abort signals and never hang"],"tags":["concurrency","rate-limit","network"],"backgroundTag":"rate-limit-exceeded","analyzedSha":"3ee70a102609e550bd2e68004bf5990a9341c851","analyzedAt":"2026-09-20T15:53:39.229Z","contentChangedAt":"2026-09-20T15:53:39.229Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}