koala73/worldmonitor · error · Error

ECCC_AGGREGATE_TOO_LARGE

ECCC_AGGREGATE_TOO_LARGE

Error message

ECCC_AGGREGATE_TOO_LARGE

What it means

All ECCC alert pages share one aggregate byte budget of ECCC_MAX_AGGREGATE_BYTES = 8 MiB across the 'issued' and 'continued' collections (each single page is also capped at ECCC_MAX_BYTES = 4 MiB). The loop checks `byteBudget.remaining <= 0` before each page and throws Error('ECCC_AGGREGATE_TOO_LARGE') to prevent the combined national alert payload from growing unbounded. readResponseLimited() also throws this when streaming bytes push the shared budget negative.

Solutions

  1. Inspect failureDetail from the aggregate error to see whether 'issued' or 'continued' (or both) exhausted the budget; a single status failing yields a partial result, not a throw.
  2. If real alert volume legitimately exceeds 8 MiB, raise ECCC_MAX_AGGREGATE_BYTES in scripts/_weather-alert-select.mjs:48, keeping ECCC_MAX_BYTES per page within the deliberate 2–4 MiB ceiling.
  3. Check whether the ECCC API started returning extra fields or verbose geometries that bloat each feature; consider trimming or compacting the payload upstream.
  4. Confirm no earlier page consumed budget twice (byteBudget is decremented inside readResponseLimited per chunk; don't decrement again at call sites).

Example fix

// before
const ECCC_MAX_AGGREGATE_BYTES = 8 * 1024 * 1024;
// after (national alert set legitimately grew)
const ECCC_MAX_AGGREGATE_BYTES = 12 * 1024 * 1024;
Defensive patterns

Strategy: try-catch

Validate before calling

// Estimate aggregate size before running the full fetch
const headIssued = await fetchFn(urlIssued + '&limit=1');
const headCont = await fetchFn(urlContinued + '&limit=1');
const matched = (await headIssued.json()).numberMatched + (await headCont.json()).numberMatched;
// ~3KB average per feature; warn if near the 8 MiB aggregate budget
if (matched * 3000 > 8 * 1024 * 1024) console.warn('ECCC aggregate byte budget at risk');

Try / catch

try {
  const result = await fetchEcccAlertFeatures({ fetchFn, userAgent });
} catch (err) {
  if (String(err.message).includes('ECCC_AGGREGATE_TOO_LARGE')) {
    reportHealth('eccc', { ok: false, reason: 'aggregate-too-large', detail: err.message });
  } else throw err;
}

Prevention

When it happens

Trigger: Fetching a page in scripts/_weather-alert-select.mjs:763 when the running total of all previously read pages has exhausted the 8 MiB shared budget (byteBudget.remaining <= 0), or inside readResponseLimited when cumulative streamed bytes drive byteBudget.remaining below 0.

Common situations: An unusually active severe-weather day where issued+continued alerts together exceed 8 MiB of JSON; an upstream API change that inflates each feature's payload size; a misconfigured caller raising maxBytes so individual pages no longer trip RESPONSE_TOO_LARGE but the aggregate does; a stub fetch in tests returning very large bodies for many pages.

Understand the failure class

Background: payload too large / request exceeds maximum size: why libraries cap bytes and how to fix oversize payloads — this error's family across 50 libraries.

Related errors

AI-assisted analysis of koala73/worldmonitor@7d06c8633d (2026-09-15). Data as JSON: /api/errors/c74d5d7c06e98cfe. Report an issue: GitHub.

Appendix: source

Thrown at scripts/_weather-alert-select.mjs:763

export async function fetchEcccAlertFeatures({
  fetchFn = globalThis.fetch,
  userAgent,
  maxBytes = ECCC_MAX_BYTES,
} = {}) {
  const byteBudget = { remaining: ECCC_MAX_AGGREGATE_BYTES };
  let pages = 0;
  const seenIds = new Set();
  const features = [];
  const failures = [];
  const failedStatuses = [];
  // Sequential paging makes the shared resource limits deterministic.
  for (const [index, status] of ECCC_LIVE_STATUSES.entries()) {
    const statusFeatures = [];
    let matched;
    try {
      do {
        if (pages >= ECCC_MAX_PAGES) throw new Error('ECCC_PAGE_LIMIT');
        if (byteBudget.remaining <= 0) throw new Error('ECCC_AGGREGATE_TOO_LARGE');
        const url = new URL(ECCC_ALERTS_URLS[index]);
        url.searchParams.set('offset', String(statusFeatures.length));
        pages += 1;
        const data = await fetchApprovedWeatherJson(url.toString(), {
          allowedHosts: [ECCC_HOST], maxBytes, fetchFn, userAgent, byteBudget,
        });
        const page = requireAlertFeatures(data);
        if (data.type !== 'FeatureCollection'
          || !Number.isSafeInteger(data.numberMatched) || data.numberMatched < 0
          || !Number.isSafeInteger(data.numberReturned) || data.numberReturned !== page.length
          || page.length > ECCC_PAGE_SIZE) {
          throw new Error('ECCC_MALFORMED_PAGE');
        }
        if (matched !== undefined && data.numberMatched !== matched) throw new Error('ECCC_COUNT_DRIFT');
        matched = data.numberMatched;
        if (statusFeatures.length + page.length > matched
          || (page.length === 0 && statusFeatures.length < matched)) {
          throw new Error('ECCC_PAGE_PROGRESS');

View on GitHub (pinned to 7d06c8633d)