koala73/worldmonitor · error · Error

ECCC_DUPLICATE_ID

ECCC_DUPLICATE_ID

Error message

ECCC_DUPLICATE_ID

What it means

ECCC_DUPLICATE_ID is thrown when two features within the paginated ECCC alert fetch resolve to the same `id`. The script maintains a `seenIds` set across all pages of one status fetch and treats any repeat as a corruption signal (e.g. the same page delivered twice), because downstream selection logic assumes ids are unique.

Solutions

1. Replay the failing status fetch and diff page contents to confirm whether the upstream really duplicates features or pagination parameters overlap
2. Verify the page-offset computation and `numberMatched` handling so consecutive pages never overlap; fix the paging loop if pages overlap
3. If ECCC legitimately emits duplicates, dedupe defensively (skip already-seen ids) instead of throwing, provided the count invariants still hold
4. Check for intermediary caches/proxies replaying responses and bypass them for a diagnostic run

Example fix

// before
if (seenIds.has(feature.id)) throw new Error('ECCC_DUPLICATE_ID');
// after
if (seenIds.has(feature.id)) {
  console.warn('ECCC duplicate feature id skipped', feature.id);
  continue;
}
seenIds.add(feature.id);

function idsAreUnique(features) {
  const ids = features.map((f) => f?.id);
  return new Set(ids).size === ids.length;
}
if (!idsAreUnique(page)) throw new Error('ECCC_DUPLICATE_ID');

null

try {
  await fetchEcccPages();
} catch (err) {
  if (err.message === 'ECCC_DUPLICATE_ID') {
    console.error('Overlapping pages detected from ECCC; resetting pagination and retrying once');
  }
}

When it happens

Trigger: During the do/while pagination loop, a feature in the current page has an id already present in `seenIds` from an earlier page or earlier in the same page — i.e. overlapping pages, a repeated first page, or the upstream API returning duplicate features.

Common situations: ECCC pagination metadata (numberMatched) is stale so pages overlap; a caching layer replays an earlier page; offset/limit parameters are computed incorrectly; the upstream returns the same feature in both issued and continued-like views within one status fetch.

Understand the failure class

Background: "This is a bug, please report it": internal invariant violations, unreachable panics, and SNH errors explained — this error's family across 47 libraries.

Related errors

• ECCC_INVALID_ID
• invalid_cursor
• invalid_limit
• ${label} HTTP 400
• Could not resolve ${JSON.stringify(echoCountryInput(raw))} t

AI-assisted analysis of koala73/worldmonitor@7d06c8633d (2026-09-15). Data as JSON: /api/errors/49e6acf85c45687b. Report an issue: GitHub.

Appendix: source

Thrown at scripts/_weather-alert-select.mjs:785

        const data = await fetchApprovedWeatherJson(url.toString(), {
          allowedHosts: [ECCC_HOST], maxBytes, fetchFn, userAgent, byteBudget,
        });
        const page = requireAlertFeatures(data);
        if (data.type !== 'FeatureCollection'
          || !Number.isSafeInteger(data.numberMatched) || data.numberMatched < 0
          || !Number.isSafeInteger(data.numberReturned) || data.numberReturned !== page.length
          || page.length > ECCC_PAGE_SIZE) {
          throw new Error('ECCC_MALFORMED_PAGE');
        }
        if (matched !== undefined && data.numberMatched !== matched) throw new Error('ECCC_COUNT_DRIFT');
        matched = data.numberMatched;
        if (statusFeatures.length + page.length > matched
          || (page.length === 0 && statusFeatures.length < matched)) {
          throw new Error('ECCC_PAGE_PROGRESS');
        }
        for (const feature of page) {
          if (typeof feature?.id !== 'string' || !feature.id.trim()) throw new Error('ECCC_INVALID_ID');
          if (seenIds.has(feature.id)) throw new Error('ECCC_DUPLICATE_ID');
          seenIds.add(feature.id);
        }
        statusFeatures.push(...page);
      } while (statusFeatures.length < matched);
      features.push(...statusFeatures);
    } catch (err) {
      failures.push(err);
      failedStatuses.push(status);
    }
  }
  if (failures.length === ECCC_LIVE_STATUSES.length) {
    const detail = failures.map((err) => err?.message || String(err)).join('; ');
    throw new Error(`ECCC issued and continued fetches both failed: ${detail}`);
  }
  // Returns an OBJECT, not a bare array, so a partial fetch cannot be consumed
  // as if it were the whole set. `issued` and `continued` are separate collections
  // and each carries alerts the other does not: `continued` is where an ONGOING
  // warning lives after its first issue. Returning just the surviving features

View on GitHub (pinned to 7d06c8633d)