{"record":{"id":"1245bf1c7899fa17","repo":"koala73/worldmonitor","slug":"eccc-malformed-page","errorCode":"ECCC_MALFORMED_PAGE","errorMessage":"ECCC_MALFORMED_PAGE","messagePattern":"ECCC_MALFORMED_PAGE","errorType":"error_code","errorClass":"Error","httpStatus":null,"severity":"error","filePath":"scripts/_weather-alert-select.mjs","lineNumber":775,"sourceCode":"  for (const [index, status] of ECCC_LIVE_STATUSES.entries()) {\n    const statusFeatures = [];\n    let matched;\n    try {\n      do {\n        if (pages >= ECCC_MAX_PAGES) throw new Error('ECCC_PAGE_LIMIT');\n        if (byteBudget.remaining <= 0) throw new Error('ECCC_AGGREGATE_TOO_LARGE');\n        const url = new URL(ECCC_ALERTS_URLS[index]);\n        url.searchParams.set('offset', String(statusFeatures.length));\n        pages += 1;\n        const data = await fetchApprovedWeatherJson(url.toString(), {\n          allowedHosts: [ECCC_HOST], maxBytes, fetchFn, userAgent, byteBudget,\n        });\n        const page = requireAlertFeatures(data);\n        if (data.type !== 'FeatureCollection'\n          || !Number.isSafeInteger(data.numberMatched) || data.numberMatched < 0\n          || !Number.isSafeInteger(data.numberReturned) || data.numberReturned !== page.length\n          || page.length > ECCC_PAGE_SIZE) {\n          throw new Error('ECCC_MALFORMED_PAGE');\n        }\n        if (matched !== undefined && data.numberMatched !== matched) throw new Error('ECCC_COUNT_DRIFT');\n        matched = data.numberMatched;\n        if (statusFeatures.length + page.length > matched\n          || (page.length === 0 && statusFeatures.length < matched)) {\n          throw new Error('ECCC_PAGE_PROGRESS');\n        }\n        for (const feature of page) {\n          if (typeof feature?.id !== 'string' || !feature.id.trim()) throw new Error('ECCC_INVALID_ID');\n          if (seenIds.has(feature.id)) throw new Error('ECCC_DUPLICATE_ID');\n          seenIds.add(feature.id);\n        }\n        statusFeatures.push(...page);\n      } while (statusFeatures.length < matched);\n      features.push(...statusFeatures);\n    } catch (err) {\n      failures.push(err);\n      failedStatuses.push(status);","sourceCodeStart":757,"sourceCodeEnd":793,"githubUrl":"https://github.com/koala73/worldmonitor/blob/7d06c8633d256c18e38133030bc3613976a96ec9/scripts/_weather-alert-select.mjs#L757-L793","documentation":"After each page fetch, the ECCC paging loop validates the OGC API Features response envelope: data.type must be 'FeatureCollection', numberMatched and numberReturned must be non-negative safe integers, numberReturned must equal the validated feature count from requireAlertFeatures(), and the page length must not exceed ECCC_PAGE_SIZE (250). Any violation throws Error('ECCC_MALFORMED_PAGE') because paging arithmetic (offset, loop termination) depends on these fields being truthful.","triggerScenarios":"The response from https://api.weather.gc.ca/collections/weather-alerts/items in scripts/_weather-alert-select.mjs:775 is a valid JSON object passing requireAlertFeatures() but violates the envelope contract: wrong type field, missing/non-integer/negative numberMatched or numberReturned, numberReturned mismatching the actual features array length, or more than 250 features in one page.","commonSituations":"The ECCC API changes its GeoJSON envelope shape or migrates to a different OGC API version; a proxy or CDN strips/alters fields; a test stub returns a plain features array without the FeatureCollection metadata; the server returns an error page that still parses as JSON; limit parameter silently ignored so a page exceeds 250 features.","solutions":["Log the offending page (or capture failureDetail from the aggregate error) and compare its shape against the OGC API Features contract: { type:'FeatureCollection', numberMatched, numberReturned, features }.","Check https://api.weather.gc.ca/collections/weather-alerts/items?f=json&limit=1 directly for an upstream schema change; update the validator if the collection legitimately moved.","If a test stub caused it, make the mock return a full FeatureCollection envelope with consistent numberMatched/numberReturned and at most ECCC_PAGE_SIZE features.","Verify requireAlertFeatures() (the earlier validation step) and this envelope check are not duplicating/contradicting constraints after a refactor."],"exampleFix":"// before: stub returns bare array\nconst data = { features: [...250 features] };\n// after: full OGC envelope matching the contract\nconst data = { type: 'FeatureCollection', numberMatched: 250, numberReturned: 250, features: [...250 features] };","handlingStrategy":"validation","validationCode":"function looksLikeOgcFeatureCollection(data) {\n  return data?.type === 'FeatureCollection'\n    && Array.isArray(data.features)\n    && Number.isSafeInteger(data.numberMatched) && data.numberMatched >= 0\n    && Number.isSafeInteger(data.numberReturned) && data.numberReturned === data.features.length\n    && data.features.length <= 250;\n}","typeGuard":"function isEcccAlertPage(data) {\n  return typeof data === 'object' && data !== null\n    && data.type === 'FeatureCollection'\n    && Number.isSafeInteger(data.numberMatched)\n    && Number.isSafeInteger(data.numberReturned)\n    && Array.isArray(data.features);\n}","tryCatchPattern":"try {\n  const result = await fetchEcccAlertFeatures({ fetchFn, userAgent });\n} catch (err) {\n  if (String(err.message).includes('ECCC_MALFORMED_PAGE')) {\n    reportHealth('eccc', { ok: false, reason: 'malformed-page', detail: err.message });\n  } else throw err;\n}","preventionTips":["In test stubs, always return the full OGC envelope (type, numberMatched, numberReturned, features) with consistent counts.","Pin/monitor the ECCC API schema; envelope changes surface as this error first.","Run a smoke fetch of one page in CI to catch upstream shape drift before production.","Validate a single page with looksLikeOgcFeatureCollection before running the multi-page fetch in tooling."],"tags":["schema-validation","geojson","ogc-api","upstream"],"backgroundTag":"unexpected-response-shape","analyzedSha":"7d06c8633d256c18e38133030bc3613976a96ec9","analyzedAt":"2026-09-15T16:44:39.439Z","contentChangedAt":"2026-09-15T16:44:39.439Z","schemaVersion":2},"datasetVersion":"2026-09-15T18:17:12.389Z"}