{"record":{"id":"255a1370aab60aaf","repo":"koala73/worldmonitor","slug":"eccc-count-drift","errorCode":"ECCC_COUNT_DRIFT","errorMessage":"ECCC_COUNT_DRIFT","messagePattern":"ECCC_COUNT_DRIFT","errorType":"error_code","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"scripts/_weather-alert-select.mjs","lineNumber":777,"sourceCode":"    let matched;\n    try {\n      do {\n        if (pages >= ECCC_MAX_PAGES) throw new Error('ECCC_PAGE_LIMIT');\n        if (byteBudget.remaining <= 0) throw new Error('ECCC_AGGREGATE_TOO_LARGE');\n        const url = new URL(ECCC_ALERTS_URLS[index]);\n        url.searchParams.set('offset', String(statusFeatures.length));\n        pages += 1;\n        const data = await fetchApprovedWeatherJson(url.toString(), {\n          allowedHosts: [ECCC_HOST], maxBytes, fetchFn, userAgent, byteBudget,\n        });\n        const page = requireAlertFeatures(data);\n        if (data.type !== 'FeatureCollection'\n          || !Number.isSafeInteger(data.numberMatched) || data.numberMatched < 0\n          || !Number.isSafeInteger(data.numberReturned) || data.numberReturned !== page.length\n          || page.length > ECCC_PAGE_SIZE) {\n          throw new Error('ECCC_MALFORMED_PAGE');\n        }\n        if (matched !== undefined && data.numberMatched !== matched) throw new Error('ECCC_COUNT_DRIFT');\n        matched = data.numberMatched;\n        if (statusFeatures.length + page.length > matched\n          || (page.length === 0 && statusFeatures.length < matched)) {\n          throw new Error('ECCC_PAGE_PROGRESS');\n        }\n        for (const feature of page) {\n          if (typeof feature?.id !== 'string' || !feature.id.trim()) throw new Error('ECCC_INVALID_ID');\n          if (seenIds.has(feature.id)) throw new Error('ECCC_DUPLICATE_ID');\n          seenIds.add(feature.id);\n        }\n        statusFeatures.push(...page);\n      } while (statusFeatures.length < matched);\n      features.push(...statusFeatures);\n    } catch (err) {\n      failures.push(err);\n      failedStatuses.push(status);\n    }\n  }","sourceCodeStart":759,"sourceCodeEnd":795,"githubUrl":"https://github.com/koala73/worldmonitor/blob/7d06c8633d256c18e38133030bc3613976a96ec9/scripts/_weather-alert-select.mjs#L759-L795","documentation":"Within one status collection ('issued' or 'continued'), the loop records numberMatched from the first page and requires every subsequent page to report the identical value. If a later page's numberMatched differs from the previously seen value, the collection changed mid-pagination, offsets become meaningless, and the loop throws Error('ECCC_COUNT_DRIFT') to avoid publishing a mixed-in-time, possibly inconsistent alert set.","triggerScenarios":"A second or later page in scripts/_weather-alert-select.mjs:777 returns a data.numberMatched value different from the value captured on the first page of the same status — i.e. alerts were created, resolved, or expired between page requests while the sequential paging loop was running.","commonSituations":"Fetching during rapidly evolving severe weather where warnings are issued/expired every few seconds; slow fetchFn (network latency) widening the window for the live collection to mutate; an upstream API bug where numberMatched fluctuates or is computed inconsistently per page; load balancers serving different backend snapshots.","solutions":["Retry the whole fetchEcccAlertFeatures() call after a short delay; drift is transient during active weather and a fresh run starts with a new baseline numberMatched.","Check whether the ECCC API offers a snapshot/transaction timestamp parameter to pin the collection during pagination; if so, include it in ECCC_ALERTS_URLS.","If drift is frequent, reduce wall-clock time per fetch (increase concurrency or page size where permitted) so pagination completes within one collection snapshot.","Verify the upstream API's numberMatched semantics haven't changed (e.g. becoming per-page instead of per-collection) and update the validator if so."],"exampleFix":null,"handlingStrategy":"retry","validationCode":null,"typeGuard":null,"tryCatchPattern":"async function fetchEcccWithRetry(opts, attempts = 3) {\n  for (let i = 0; i < attempts; i++) {\n    try {\n      return await fetchEcccAlertFeatures(opts);\n    } catch (err) {\n      if (!String(err.message).includes('ECCC_COUNT_DRIFT') || i === attempts - 1) throw err;\n      await new Promise((r) => setTimeout(r, 1000 * (i + 1)));\n    }\n  }\n}","preventionTips":["Schedule fetches away from peak alert churn where possible; drift is a live-data race.","Treat a single drifted status as a partial result (result.partial) rather than a hard failure.","Retry with backoff — a fresh run captures a new collection snapshot with a consistent numberMatched.","If drift recurs persistently, investigate upstream load-balancer snapshot consistency rather than loosening the check."],"tags":["pagination","consistency","upstream","race-condition"],"backgroundTag":"unexpected-response-shape","analyzedSha":"7d06c8633d256c18e38133030bc3613976a96ec9","analyzedAt":"2026-09-15T16:44:39.439Z","contentChangedAt":"2026-09-15T16:44:39.439Z","schemaVersion":2},"datasetVersion":"2026-09-15T18:17:12.389Z"}