koala73/worldmonitor · error · Error

crawlable corpus captured dated country developments for ${d

Error message

crawlable corpus captured dated country developments for ${developmentsPageCount} of ${indexedCountryPageCount} indexed country pages; expected at least ${floor}${ratioOverride != null ? ` (operator override ${ratioOverride})` : countryIndexAttempted ? ' for an index-era capture' : ''}. A snapshot that carries developments should cover far more than this — check the ${countryIndexAttempted ? 'per-country index top-up (coverage.developmentsCountryIndex) and the digest match ' : 'digest match and the freeze service key '}before republishing, or publish a measured-but-lower week with ${DEVELOPMENTS_COVERAGE_RATIO_ENV}.

What it means

Tripwire enforcing that a corpus snapshot which includes dated country developments covers at least a minimum share (MIN_DEVELOPMENTS_COVERAGE_RATIO, or a higher floor when the per-country index top-up ran; overridable via CRAWLABLE_DEVELOPMENTS_COVERAGE_RATIO) of indexed country pages, with a floor of at least 1 page. Falling below the ceil(count * ratio) floor throws, blocking republish of an under-covered snapshot.

Solutions

Inspect coverage.developmentsCountryIndex and verify the per-country index top-up ran and matched the digest.
Check the freeze service key / upstream developments feed for outages or empty responses.
If the capture is legitimately lower (slow week), set CRAWLABLE_DEVELOPMENTS_COVERAGE_RATIO to a measured, valid ratio and re-run.
Re-capture the snapshot after fixing the upstream source and republish.

Example fix

// before (block republish, no override)
npm run build:crawlable-corpus
// after (legitimately low week, measured override)
CRAWLABLE_DEVELOPMENTS_COVERAGE_RATIO=0.35 npm run build:crawlable-corpus

Defensive patterns

Strategy: validation

Validate before calling

const floor = Math.max(1, Math.ceil(indexed * ratio));
if (developments < floor) throw new Error(`developments coverage ${developments}/${indexed} below floor ${floor}`);

Try / catch

try { assertDevelopmentsCoverage({...}); } catch (e) { console.error(e.message); /* inspect coverage.developmentsCountryIndex / freeze service before overriding */ }

Prevention

Monitor the developments freeze service and index top-up health before capture.
Only use CRAWLABLE_DEVELOPMENTS_COVERAGE_RATIO with a measured, justified value.
Alert on coverage.developmentsCountryIndex drops in build telemetry.

When it happens

Trigger: Running the crawlable-corpus build with carriesDevelopments=true when developmentsPageCount is below Math.max(1, ceil(indexedCountryPageCount * ratio)) — e.g. the freeze service returned few items, the digest changed so few pages matched, or the per-country index top-up failed silently.

Common situations: Upstream news/freeze service outage during capture; stale digest causing the index top-up to miss; a genuinely slow news week with no override configured; coverage ratio defaults raised in a recent release.

Understand the failure class

Background: Schema validation failed / invalid input schema: payload rejected because its shape doesn't match the expected schema — this error's family across 28 libraries.

Related errors

AI-assisted analysis of koala73/worldmonitor@7d06c8633d (2026-09-15). Data as JSON: /api/errors/706211637964f1e2. Report an issue: GitHub.

Appendix: source

Thrown at scripts/build-crawlable-corpus.mjs:3667

// Pipeline tripwire decision (#7615, retuned in #7620 follow-up), exported for
// tests: the per-page guard proves frozen items render; this proves the capture
// did not collapse. `countryIndexAttempted` is the snapshot's own declaration
// (it carries coverage.developmentsCountryIndex), so a snapshot frozen before
// the index existed keeps the collapse floor.
export function assertDevelopmentsCoverage({
  carriesDevelopments,
  developmentsPageCount,
  indexedCountryPageCount,
  countryIndexAttempted = false,
  ratioOverride = null,
}) {
  if (!carriesDevelopments) return;
  const ratio = ratioOverride
    ?? (countryIndexAttempted ? MIN_DEVELOPMENTS_COVERAGE_RATIO_WITH_COUNTRY_INDEX : MIN_DEVELOPMENTS_COVERAGE_RATIO);
  const floor = Math.max(1, Math.ceil(indexedCountryPageCount * ratio));
  if (developmentsPageCount < floor) {
    throw new Error(
      `crawlable corpus captured dated country developments for ${developmentsPageCount} `
      + `of ${indexedCountryPageCount} indexed country pages; expected at least ${floor}`
      + (ratioOverride != null ? ` (operator override ${ratioOverride})` : countryIndexAttempted ? ' for an index-era capture' : '')
      + '. A snapshot that carries developments should cover far more than this — check the '
      + (countryIndexAttempted
        ? 'per-country index top-up (coverage.developmentsCountryIndex) and the digest match '
        : 'digest match and the freeze service key ')
      + `before republishing, or publish a measured-but-lower week with ${DEVELOPMENTS_COVERAGE_RATIO_ENV}.`,
    );
  }
}

/** Whether a frozen snapshot's freeze attempted the per-country article index top-up (#7748), whatever it answered. */
export function snapshotAttemptedCountryIndex(livePulse) {
  const record = livePulse?.coverage?.developmentsCountryIndex;
  return Boolean(record) && typeof record === 'object' && typeof record.state === 'string';
}

View on GitHub (pinned to 7d06c8633d)