koala73/worldmonitor · error · Error

ECCC_PAGE_LIMIT

ECCC_PAGE_LIMIT

Error message

ECCC_PAGE_LIMIT

What it means

fetchEcccAlertFeatures() pages the ECCC weather-alerts OGC API (api.weather.gc.ca) with a hard cap of ECCC_MAX_PAGES = 8 pages shared across both 'issued' and 'continued' status collections. Before each page request it checks `pages >= ECCC_MAX_PAGES` and throws Error('ECCC_PAGE_LIMIT') to stop an unbounded paging loop. Thrown when the paged collections legitimately need more than 8 pages of 250 features (2000 alerts), or when pagination makes no progress so the loop never terminates.

Solutions

1. Check the actual alert volume: query https://api.weather.gc.ca/collections/weather-alerts/items?f=json&status_en=issued&limit=1 and inspect numberMatched; if it approaches 2000, raise ECCC_MAX_PAGES in scripts/_weather-alert-select.mjs:49.
2. Verify the server honors limit=250 by checking data.numberReturned per page; if pages return few items, the ECCC API changed its paging behavior and ECCC_PAGE_SIZE/maxBytes tuning is needed.
3. If a single status trips the limit, note fetchEcccAlertFeatures degrades gracefully: only when BOTH statuses fail does it throw the aggregate error; inspect failureDetail in the thrown message to see which statuses failed and why.
4. For tests, ensure the mocked fetch respects offset/limit paging and returns numberMatched consistent with finite pages so the loop terminates within 8 iterations.

Example fix

// before
const ECCC_MAX_PAGES = 8;
// after (if alert volume legitimately exceeds 2000)
const ECCC_MAX_PAGES = 16;

// Pre-check alert volume before paging
const probe = await fetchFn('https://api.weather.gc.ca/collections/weather-alerts/items?f=json&status_en=issued&limit=1');
const probeJson = await probe.json();
const expectedPages = Math.ceil((probeJson.numberMatched ?? 0) / 250);
if (expectedPages > 8) console.warn('ECCC_PAGE_LIMIT risk: needs', expectedPages, 'pages');

function hasFinitePageEstimate(envelope) {
  return typeof envelope === 'object' && envelope !== null
    && Number.isSafeInteger(envelope.numberMatched)
    && Math.ceil(envelope.numberMatched / 250) <= 8;
}

try {
  const result = await fetchEcccAlertFeatures({ fetchFn, userAgent });
} catch (err) {
  if (String(err.message).includes('ECCC_PAGE_LIMIT')) {
    // Page budget exhausted: surface partial-data health, do not publish truncated set
    reportHealth('eccc', { ok: false, reason: 'page-limit', detail: err.message });
  } else throw err;
}

When it happens

Trigger: The `do { ... } while (statusFeatures.length < matched)` loop in scripts/_weather-alert-select.mjs:762 requests more than 8 pages in total across the 'issued' and 'continued' collections — i.e. numberMatched exceeds 2000 features for a status, or the offset-based paging keeps returning features so the loop continues past the page budget.

Common situations: A severe weather day where Canada-wide issued+continued alerts exceed 2000; an upstream API regression where the `limit` parameter is ignored (server returns fewer than ECCC_PAGE_SIZE=250 per page, forcing more pages); a broken `matched` value that never satisfies the loop-exit condition, causing endless paging; tests using a fetch stub that always reports more results.

Understand the failure class

Background: "value must be between 0 and 1" / "out of range" / "must not be negative" errors: fixing range-validation failures across open-source libraries — this error's family across 42 libraries.

Related errors

• ECCC_AGGREGATE_TOO_LARGE
• Sentry issues pagination exceeded ${MAX_PAGES} pages
• Invalid scorecard bloc selection.
• invalid_cursor
• invalid_limit

AI-assisted analysis of koala73/worldmonitor@7d06c8633d (2026-09-15). Data as JSON: /api/errors/a12c4a370adbfe88. Report an issue: GitHub.

Appendix: source

Thrown at scripts/_weather-alert-select.mjs:762

 */
export async function fetchEcccAlertFeatures({
  fetchFn = globalThis.fetch,
  userAgent,
  maxBytes = ECCC_MAX_BYTES,
} = {}) {
  const byteBudget = { remaining: ECCC_MAX_AGGREGATE_BYTES };
  let pages = 0;
  const seenIds = new Set();
  const features = [];
  const failures = [];
  const failedStatuses = [];
  // Sequential paging makes the shared resource limits deterministic.
  for (const [index, status] of ECCC_LIVE_STATUSES.entries()) {
    const statusFeatures = [];
    let matched;
    try {
      do {
        if (pages >= ECCC_MAX_PAGES) throw new Error('ECCC_PAGE_LIMIT');
        if (byteBudget.remaining <= 0) throw new Error('ECCC_AGGREGATE_TOO_LARGE');
        const url = new URL(ECCC_ALERTS_URLS[index]);
        url.searchParams.set('offset', String(statusFeatures.length));
        pages += 1;
        const data = await fetchApprovedWeatherJson(url.toString(), {
          allowedHosts: [ECCC_HOST], maxBytes, fetchFn, userAgent, byteBudget,
        });
        const page = requireAlertFeatures(data);
        if (data.type !== 'FeatureCollection'
          || !Number.isSafeInteger(data.numberMatched) || data.numberMatched < 0
          || !Number.isSafeInteger(data.numberReturned) || data.numberReturned !== page.length
          || page.length > ECCC_PAGE_SIZE) {
          throw new Error('ECCC_MALFORMED_PAGE');
        }
        if (matched !== undefined && data.numberMatched !== matched) throw new Error('ECCC_COUNT_DRIFT');
        matched = data.numberMatched;
        if (statusFeatures.length + page.length > matched
          || (page.length === 0 && statusFeatures.length < matched)) {

View on GitHub (pinned to 7d06c8633d)