koala73/worldmonitor · error

RESPONSE_TOO_LARGE

RESPONSE_TOO_LARGE

Error message

RESPONSE_TOO_LARGE

What it means

fetchMndViaProxy enforces the Taiwan MND source contract's maxResponseBytes (CROSS_STRAIT_SOURCE_CONTRACTS.taiwanMnd.maxResponseBytes) on proxy-fetched bodies. Even with a valid status and Buffer, if result.buffer.byteLength exceeds the cap the body is rejected with RESPONSE_TOO_LARGE so an oversized or hostile response is never parsed downstream.

Solutions

1. Log result.buffer.byteLength and the contract's maxResponseBytes to see by how much the response exceeded the cap.
2. Make the proxy honor the maxResponseBytes option by truncating/aborting the download at the cap instead of buffering the entire body.
3. If the real MND responses have legitimately grown, raise taiwanMnd.maxResponseBytes in CROSS_STRAIT_SOURCE_CONTRACTS to a justified value.
4. Check whether the proxy is accidentally decompressing or re-fetching a much larger resource than intended (wrong URL, redirects to big assets).

Example fix

// before: proxy buffers whole body, ignoring cap
const buffer = Buffer.concat(chunks);
return { status, buffer };

// after: enforce the cap inside the proxy as well
if (total > maxResponseBytes) {
  await reader.cancel().catch(() => {});
  throw new Error('RESPONSE_TOO_LARGE');
}
return { status, buffer: Buffer.concat(chunks, total) };

// Guard before returning from the proxy layer
const cap = CROSS_STRAIT_SOURCE_CONTRACTS.taiwanMnd.maxResponseBytes;
if (buffer.byteLength > cap) throw new Error(`RESPONSE_TOO_LARGE:${buffer.byteLength}>${cap}`);

try {
  return await fetchMndViaProxy(input, init, proxyConfig, proxyRequestFn);
} catch (error) {
  if (error?.message === 'RESPONSE_TOO_LARGE') {
    logger.warn({ url: String(input) }, 'MND proxy response exceeded maxResponseBytes');
    return null; // treat as unavailable, do not parse
  }
  throw error;
}

When it happens

Trigger: The MND proxy returns a 2xx Buffer body whose byteLength exceeds taiwanMnd.maxResponseBytes — e.g. the proxy ignores the maxResponseBytes option passed in init, the endpoint unexpectedly returns a huge page/binary, or the configured cap was lowered below typical response sizes.

Common situations: Proxy implementation streams the whole body without honoring maxResponseBytes truncation; upstream MND site change dramatically inflates page size; misconfigured/undersized maxResponseBytes in the source contract; proxy returning a compressed or inflated payload larger than expected.

Understand the failure class

Background: payload too large / request exceeds maximum size: why libraries cap bytes and how to fix oversize payloads — this error's family across 50 libraries.

Related errors

• MND_PROXY_RESPONSE_INVALID
• IMD_RESPONSE_TOO_LARGE:${bytes}
• HTTP ${resp.status}
• HTTP ${resp.status}
• Webhook URL is not a valid URL

AI-assisted analysis of koala73/worldmonitor@7d06c8633d (2026-09-15). Data as JSON: /api/errors/cd305a77cdf84f27. Report an issue: GitHub.

Appendix: source

Thrown at scripts/cross-strait-activity/adapters.mjs:1734

  return text;
}

async function fetchMndViaProxy(input, init, proxyConfig, proxyRequestFn) {
  const maxResponseBytes = CROSS_STRAIT_SOURCE_CONTRACTS.taiwanMnd.maxResponseBytes;
  const result = await proxyRequestFn(String(input), proxyConfig, {
    headers: init.headers,
    maxResponseBytes,
    timeoutMs: REQUEST_TIMEOUT_MS,
    signal: init.signal,
  });
  if (!Number.isInteger(result?.status) || result.status < 200 || result.status > 599) {
    throw new Error('MND_PROXY_RESPONSE_INVALID');
  }
  if (result.status >= 300) {
    return new Response(null, { status: result.status });
  }
  if (!Buffer.isBuffer(result.buffer)) throw new Error('MND_PROXY_RESPONSE_INVALID');
  if (result.buffer.byteLength > maxResponseBytes) throw new Error('RESPONSE_TOO_LARGE');
  return new Response(result.status === 204 || result.status === 205 ? null : result.buffer, {
    status: result.status,
  });
}

function shouldProxyJapanModFailure(error) {
  const code = errorCode(error);
  if (code === 'SOURCE_ERROR' || code === 'TIMEOUT' || code === 'JMOD_INDEX_EMPTY') return true;
  const status = Number(/^HTTP_(\d{3})$/u.exec(code)?.[1]);
  return status === 403
    || status === 408
    || status === 425
    || status === 429
    || status >= 500;
}

async function fetchJapanModViaConfiguredProxy(input, init, {
  proxyUrl,

View on GitHub (pinned to 7d06c8633d)