{"record":{"id":"49e6acf85c45687b","repo":"koala73/worldmonitor","slug":"eccc-duplicate-id","errorCode":"ECCC_DUPLICATE_ID","errorMessage":"ECCC_DUPLICATE_ID","messagePattern":"ECCC_DUPLICATE_ID","errorType":"error_code","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"scripts/_weather-alert-select.mjs","lineNumber":785,"sourceCode":"        const data = await fetchApprovedWeatherJson(url.toString(), {\n          allowedHosts: [ECCC_HOST], maxBytes, fetchFn, userAgent, byteBudget,\n        });\n        const page = requireAlertFeatures(data);\n        if (data.type !== 'FeatureCollection'\n          || !Number.isSafeInteger(data.numberMatched) || data.numberMatched < 0\n          || !Number.isSafeInteger(data.numberReturned) || data.numberReturned !== page.length\n          || page.length > ECCC_PAGE_SIZE) {\n          throw new Error('ECCC_MALFORMED_PAGE');\n        }\n        if (matched !== undefined && data.numberMatched !== matched) throw new Error('ECCC_COUNT_DRIFT');\n        matched = data.numberMatched;\n        if (statusFeatures.length + page.length > matched\n          || (page.length === 0 && statusFeatures.length < matched)) {\n          throw new Error('ECCC_PAGE_PROGRESS');\n        }\n        for (const feature of page) {\n          if (typeof feature?.id !== 'string' || !feature.id.trim()) throw new Error('ECCC_INVALID_ID');\n          if (seenIds.has(feature.id)) throw new Error('ECCC_DUPLICATE_ID');\n          seenIds.add(feature.id);\n        }\n        statusFeatures.push(...page);\n      } while (statusFeatures.length < matched);\n      features.push(...statusFeatures);\n    } catch (err) {\n      failures.push(err);\n      failedStatuses.push(status);\n    }\n  }\n  if (failures.length === ECCC_LIVE_STATUSES.length) {\n    const detail = failures.map((err) => err?.message || String(err)).join('; ');\n    throw new Error(`ECCC issued and continued fetches both failed: ${detail}`);\n  }\n  // Returns an OBJECT, not a bare array, so a partial fetch cannot be consumed\n  // as if it were the whole set. `issued` and `continued` are separate collections\n  // and each carries alerts the other does not: `continued` is where an ONGOING\n  // warning lives after its first issue. Returning just the surviving features","sourceCodeStart":767,"sourceCodeEnd":803,"githubUrl":"https://github.com/koala73/worldmonitor/blob/7d06c8633d256c18e38133030bc3613976a96ec9/scripts/_weather-alert-select.mjs#L767-L803","documentation":"ECCC_DUPLICATE_ID is thrown when two features within the paginated ECCC alert fetch resolve to the same `id`. The script maintains a `seenIds` set across all pages of one status fetch and treats any repeat as a corruption signal (e.g. the same page delivered twice), because downstream selection logic assumes ids are unique.","triggerScenarios":"During the do/while pagination loop, a feature in the current page has an id already present in `seenIds` from an earlier page or earlier in the same page — i.e. overlapping pages, a repeated first page, or the upstream API returning duplicate features.","commonSituations":"ECCC pagination metadata (numberMatched) is stale so pages overlap; a caching layer replays an earlier page; offset/limit parameters are computed incorrectly; the upstream returns the same feature in both issued and continued-like views within one status fetch.","solutions":["Replay the failing status fetch and diff page contents to confirm whether the upstream really duplicates features or pagination parameters overlap","Verify the page-offset computation and `numberMatched` handling so consecutive pages never overlap; fix the paging loop if pages overlap","If ECCC legitimately emits duplicates, dedupe defensively (skip already-seen ids) instead of throwing, provided the count invariants still hold","Check for intermediary caches/proxies replaying responses and bypass them for a diagnostic run"],"exampleFix":"// before\nif (seenIds.has(feature.id)) throw new Error('ECCC_DUPLICATE_ID');\n// after\nif (seenIds.has(feature.id)) {\n  console.warn('ECCC duplicate feature id skipped', feature.id);\n  continue;\n}\nseenIds.add(feature.id);","handlingStrategy":"validation","validationCode":"function idsAreUnique(features) {\n  const ids = features.map((f) => f?.id);\n  return new Set(ids).size === ids.length;\n}\nif (!idsAreUnique(page)) throw new Error('ECCC_DUPLICATE_ID');","typeGuard":"null","tryCatchPattern":"try {\n  await fetchEcccPages();\n} catch (err) {\n  if (err.message === 'ECCC_DUPLICATE_ID') {\n    console.error('Overlapping pages detected from ECCC; resetting pagination and retrying once');\n  }\n}","preventionTips":["Assert page boundaries never overlap given the offset/limit math","Compare numberMatched against accumulated count each iteration","Bypass caches when diagnosing duplicate page delivery","Decide explicitly whether duplicates should skip-and-warn rather than hard-fail"],"tags":["validation","pagination","duplicate-data","weather-api"],"backgroundTag":"internal-invariant-violation","analyzedSha":"7d06c8633d256c18e38133030bc3613976a96ec9","analyzedAt":"2026-09-15T16:44:39.439Z","contentChangedAt":"2026-09-15T16:44:39.439Z","schemaVersion":2},"datasetVersion":"2026-09-15T18:17:12.389Z"}