koala73/worldmonitor · error · Error

ECCC_PAGE_PROGRESS

ECCC_PAGE_PROGRESS

Error message

ECCC_PAGE_PROGRESS

What it means

The paging loop requires monotonic, truthful progress: accumulated features plus the current page must not exceed numberMatched, and a zero-length page is only allowed once all matched features have been collected (statusFeatures.length === matched). A page that overshoots the declared total, or an empty page returned while features are still outstanding, indicates broken pagination semantics, so Error('ECCC_PAGE_PROGRESS') is thrown rather than risking a truncated or looped result.

Solutions

1. Query the collection manually with successive offsets to reproduce which page breaks: ...?limit=250&offset=N; confirm whether the upstream server skips or repeats items.
2. Retry the fetch — if alerts changed mid-pagination causing the overshoot, a fresh run with a fresh numberMatched usually succeeds.
3. If the ECCC API regressed on offset paging, consider switching the pagination key (sortby=feature_id is already set) or paging on the returned feature ids instead of raw offsets.
4. Fix test stubs to implement offset correctly: return exactly `limit` features per page from the sorted dataset and an empty page only at/after the end.

Example fix

// before: stub ignores offset and always returns page 1
const page = allFeatures.slice(0, limit);
// after: honor offset so progress is monotonic
const page = allFeatures.slice(offset, offset + limit);

function pageIsProgressive(accumulated, page, matched) {
  if (accumulated + page.length > matched) return false;
  if (page.length === 0 && accumulated < matched) return false;
  return true;
}

async function fetchEcccSafe(opts) {
  try {
    return await fetchEcccAlertFeatures(opts);
  } catch (err) {
    if (String(err.message).includes('ECCC_PAGE_PROGRESS')) {
      // Upstream pagination skipped/overshot; one retry usually lands on a consistent snapshot
      return fetchEcccAlertFeatures(opts);
    }
    throw err;
  }
}

When it happens

Trigger: In scripts/_weather-alert-select.mjs:781: (a) statusFeatures.length + page.length > matched — the server returned more features than numberMatched promised (e.g. duplicates pushed by a concurrent mutation or an offset that restarted), or (b) page.length === 0 while statusFeatures.length < matched — the server returned an empty page before the collection was fully drained.

Common situations: An upstream OGC API regression where offset paging returns empty pages mid-collection; alerts expiring between pages so the offset window shifts and the final page over-collects relative to the (stale) numberMatched; a proxy caching an older page and replaying it, breaking forward progress; a test stub with incorrect offset handling.

Related errors

• ECCC_COUNT_DRIFT
• ECCC_MALFORMED_PAGE
• relay returned ${resp.status}
• Physical divergence composite does not match its member read
• Wikidata request failed with status ${response.status}

AI-assisted analysis of koala73/worldmonitor@7d06c8633d (2026-09-15). Data as JSON: /api/errors/29a7df874f6e834f. Report an issue: GitHub.

Appendix: source

Thrown at scripts/_weather-alert-select.mjs:781

        if (byteBudget.remaining <= 0) throw new Error('ECCC_AGGREGATE_TOO_LARGE');
        const url = new URL(ECCC_ALERTS_URLS[index]);
        url.searchParams.set('offset', String(statusFeatures.length));
        pages += 1;
        const data = await fetchApprovedWeatherJson(url.toString(), {
          allowedHosts: [ECCC_HOST], maxBytes, fetchFn, userAgent, byteBudget,
        });
        const page = requireAlertFeatures(data);
        if (data.type !== 'FeatureCollection'
          || !Number.isSafeInteger(data.numberMatched) || data.numberMatched < 0
          || !Number.isSafeInteger(data.numberReturned) || data.numberReturned !== page.length
          || page.length > ECCC_PAGE_SIZE) {
          throw new Error('ECCC_MALFORMED_PAGE');
        }
        if (matched !== undefined && data.numberMatched !== matched) throw new Error('ECCC_COUNT_DRIFT');
        matched = data.numberMatched;
        if (statusFeatures.length + page.length > matched
          || (page.length === 0 && statusFeatures.length < matched)) {
          throw new Error('ECCC_PAGE_PROGRESS');
        }
        for (const feature of page) {
          if (typeof feature?.id !== 'string' || !feature.id.trim()) throw new Error('ECCC_INVALID_ID');
          if (seenIds.has(feature.id)) throw new Error('ECCC_DUPLICATE_ID');
          seenIds.add(feature.id);
        }
        statusFeatures.push(...page);
      } while (statusFeatures.length < matched);
      features.push(...statusFeatures);
    } catch (err) {
      failures.push(err);
      failedStatuses.push(status);
    }
  }
  if (failures.length === ECCC_LIVE_STATUSES.length) {
    const detail = failures.map((err) => err?.message || String(err)).join('; ');
    throw new Error(`ECCC issued and continued fetches both failed: ${detail}`);
  }

View on GitHub (pinned to 7d06c8633d)