{"record":{"id":"ed4f57cf49615ae0","repo":"koala73/worldmonitor","slug":"jodi-gas-catalog-has-no-valid-publication","errorCode":null,"errorMessage":"JODI Gas catalog has no valid publication","messagePattern":"JODI Gas catalog has no valid publication","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"scripts/seed-jodi-gas.mjs","lineNumber":267,"sourceCode":"    if (!filename || entryName === filename || entryName.endsWith('/' + filename)) {\n      if ((flags & 0x08) && compSize === 0) {\n        throw new Error(`JODI Gas ZIP: entry uses data-descriptor (bit 3) — compSize unknown in local header`);\n      }\n      return { dataOffset, compSize, compression, entryName };\n    }\n    // If bit 3 is set, compSize in local header is 0 — fall back to byte scan\n    if ((flags & 0x08) && compSize === 0) { offset++; continue; }\n    offset = dataOffset + compSize;\n  }\n  return null;\n}\n\n// Publication IDs identify specific releases. Follow the same catalog as the\n// official download page; a successful fetch of an old ID never finds new data.\nfunction resolveGasArchiveUrl(catalog) {\n  if (!Number.isSafeInteger(catalog?.publicationId) || catalog.publicationId <= 0\n    || !Array.isArray(catalog.files)) {\n    throw new Error('JODI Gas catalog has no valid publication');\n  }\n  const csvFiles = catalog.files.filter(file => file?.format === 'CSV' && file.ignore === false);\n  if (csvFiles.length !== 1 || csvFiles[0].filename !== 'GAS_world_NewFormat.zip') {\n    throw new Error('JODI Gas catalog must contain one supported CSV archive');\n  }\n  return `${ARCHIVE_ROOT}/${catalog.publicationId}/${csvFiles[0].filename}`;\n}\n\nasync function fetchAndParseCsv() {\n  // Discovery and archive download share the existing total request budget.\n  const signal = AbortSignal.timeout(120_000);\n  const catalogResponse = await fetch(CATALOG_URL, {\n    headers: { 'User-Agent': CHROME_UA },\n    signal,\n    redirect: 'error',\n  });\n  if (!catalogResponse.ok) throw new Error(`JODI Gas catalog fetch failed: HTTP ${catalogResponse.status}`);\n  const zipUrl = resolveGasArchiveUrl(await catalogResponse.json());","sourceCodeStart":249,"sourceCodeEnd":285,"githubUrl":"https://github.com/koala73/worldmonitor/blob/e586b8b4b80f595aa7ece295eec10d76f2921240/scripts/seed-jodi-gas.mjs#L249-L285","documentation":"resolveGasArchiveUrl() validates the JODI Gas catalog JSON before using it: it requires a positive safe-integer publicationId and an array files. If either is missing or of the wrong type/shape, it throws 'JODI Gas catalog has no valid publication'. This detects upstream catalog format changes or partial responses before any URL is constructed.","triggerScenarios":"The catalog endpoint returns JSON without publicationId (schema change), with a string or float id, or with files absent/null; a proxy or error page returns JSON that isn't the real catalog.","commonSituations":"JODI publishes a new catalog schema, an intermediate/proxy replaces the response, or a cached/stale catalog snapshot is validated.","solutions":["Inspect the raw catalog JSON from CATALOG_URL and compare against the expected shape (publicationId as positive integer, files array)","Update resolveGasArchiveUrl() and seed-jodi-gas.mjs if the JODI catalog schema changed","Re-run after confirming the endpoint returns the genuine catalog (no proxy/error JSON)","Check seed-meta/cache for a stale catalog copy being validated"],"exampleFix":"// before\nthrow new Error('JODI Gas catalog has no valid publication');\n// after (surface what actually came back)\nthrow new Error(`JODI Gas catalog has no valid publication: ${JSON.stringify(catalog).slice(0, 200)}`);","handlingStrategy":"type-guard","validationCode":"if (!Number.isSafeInteger(catalog?.publicationId) || catalog.publicationId <= 0 || !Array.isArray(catalog.files)) {\n  throw new Error('catalog shape invalid before calling resolveGasArchiveUrl');\n}","typeGuard":"function hasValidPublication(catalog) {\n  return Number.isSafeInteger(catalog?.publicationId) && catalog.publicationId > 0 && Array.isArray(catalog?.files);\n}","tryCatchPattern":"try {\n  const zipUrl = resolveGasArchiveUrl(catalog);\n} catch (err) {\n  if (err.message.includes('no valid publication')) console.error('Catalog JSON shape changed; inspect the raw payload');\n  throw err;\n}","preventionTips":["Validate the catalog JSON shape right after fetching, before use","Snapshot the raw catalog response when validation fails for diagnosis","Watch for upstream schema changes and update the validator accordingly"],"tags":["validation","schema","external-api","data-seeding"],"backgroundTag":"schema-validation-failed","analyzedSha":"e586b8b4b80f595aa7ece295eec10d76f2921240","analyzedAt":"2026-09-22T01:50:49.965Z","contentChangedAt":"2026-09-22T01:50:49.965Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}