koala73/worldmonitor · error

IMD_RESPONSE_TOO_LARGE:${bytes}

Error message

IMD_RESPONSE_TOO_LARGE:${bytes}

What it means

readBoundedJsonResponse in scripts/lib/imd-cyclone-marine.mjs caps IMD (India Meteorological Department) JSON responses at maxBytes (default IMD_MAX_BYTES). When the accumulated body — either from response.text() on a non-streaming body or from summing stream chunks — exceeds the cap, it throws IMD_RESPONSE_TOO_LARGE:<byteCount>, embedding the actual size in the message so callers can see the overage.

Solutions

1. Read the byte count in the error message (e.g. IMD_RESPONSE_TOO_LARGE:5242880) and compare it with IMD_MAX_BYTES / the maxBytes option to size the overage.
2. If the product's responses legitimately exceed the cap, raise maxBytes (or IMD_MAX_BYTES) to a justified value when calling fetchApprovedImdJson.
3. Check whether the endpoint supports pagination/filtering (smaller product windows) and request that instead of the bulk payload.
4. Verify no proxy/gateway is inflating responses (e.g. concatenating payloads or disabling compression) and that you are hitting the intended product URL.

Example fix

// before: default cap rejects a legitimately larger new IMD product
const data = await fetchApprovedImdJson(url);

// after: raise the cap for this product with a justified limit
const data = await fetchApprovedImdJson(url, { maxBytes: 10 * 1024 * 1024 });

// Pre-flight: check content-length before reading when available
const len = Number(response.headers.get('content-length'));
if (Number.isFinite(len) && len > maxBytes) {
  throw new Error(`IMD_RESPONSE_TOO_LARGE:${len}`); // avoid reading at all
}

try {
  const data = await fetchApprovedImdJson(url, { maxBytes });
  return data;
} catch (error) {
  if (String(error?.message).startsWith('IMD_RESPONSE_TOO_LARGE:')) {
    const bytes = Number(error.message.split(':')[1]);
    logger.warn({ url, bytes, maxBytes }, 'IMD response exceeded cap');
    return null; // skip this product
  }
  throw error;
}

When it happens

Trigger: An IMD endpoint (cyclone, marine bulletin, or forecast product JSON) returns more bytes than IMD_MAX_BYTES; a product feed returns the full archive instead of a single page; the caller passed a custom maxBytes below the typical payload size in fetchApprovedImdJson options.

Common situations: IMD API silently returns an unpaginated bulk response during severe-weather events; proxy or gateway decompression inflates payload size; developer lowers maxBytes for testing and forgets to restore it; new IMD product added whose responses exceed the default cap.

Understand the failure class

Background: payload too large / request exceeds maximum size: why libraries cap bytes and how to fix oversize payloads — this error's family across 50 libraries.

Related errors

• RESPONSE_TOO_LARGE
• IMD_RESPONSE_TOO_LARGE:${size}
• Exa returned malformed structured summary for ${url}
• Redis returned malformed JSON
• --args must be valid JSON: ${err.message}

AI-assisted analysis of koala73/worldmonitor@7d06c8633d (2026-09-15). Data as JSON: /api/errors/7207f37ad1f40c29. Report an issue: GitHub.

Appendix: source

Thrown at scripts/lib/imd-cyclone-marine.mjs:732

  });
}

function decorateSnapshotSurfaces(snapshot) {
  return {
    ...snapshot,
    cycloneEvents: cycloneEventsFromSnapshot(snapshot),
    portAlerts: weatherAlertsFromSnapshot(snapshot),
    marineBulletins: marineBulletinsFromSnapshot(snapshot),
  };
}

async function readBoundedJsonResponse(response, maxBytes = IMD_MAX_BYTES) {
  const chunks = [];
  let size = 0;
  if (!response.body || typeof response.body.getReader !== 'function') {
    const text = await response.text();
    const bytes = Buffer.byteLength(text, 'utf8');
    if (bytes > maxBytes) throw new Error(`IMD_RESPONSE_TOO_LARGE:${bytes}`);
    return JSON.parse(text);
  }
  const reader = response.body.getReader();
  for (;;) {
    const { done, value } = await reader.read();
    if (done) break;
    size += value.byteLength;
    if (size > maxBytes) throw new Error(`IMD_RESPONSE_TOO_LARGE:${size}`);
    chunks.push(value);
  }
  return JSON.parse(new TextDecoder().decode(Buffer.concat(chunks.map((chunk) => Buffer.from(chunk)))));
}

export async function fetchApprovedImdJson(url, {
  fetchFn = globalThis.fetch,
  userAgent = CHROME_UA,
  maxBytes = IMD_MAX_BYTES,
  timeoutMs = IMD_TIMEOUT_MS,

View on GitHub (pinned to 7d06c8633d)