{"record":{"id":"3b75b1ed1a7a2627","repo":"koala73/worldmonitor","slug":"jodi-gas-catalog-fetch-failed-http-catalogresponse-status","errorCode":null,"errorMessage":"JODI Gas catalog fetch failed: HTTP ${catalogResponse.status}","messagePattern":"JODI Gas catalog fetch failed: HTTP (.+?)","errorType":"exception","errorClass":null,"httpStatus":null,"severity":"error","filePath":"scripts/seed-jodi-gas.mjs","lineNumber":284,"sourceCode":"    || !Array.isArray(catalog.files)) {\n    throw new Error('JODI Gas catalog has no valid publication');\n  }\n  const csvFiles = catalog.files.filter(file => file?.format === 'CSV' && file.ignore === false);\n  if (csvFiles.length !== 1 || csvFiles[0].filename !== 'GAS_world_NewFormat.zip') {\n    throw new Error('JODI Gas catalog must contain one supported CSV archive');\n  }\n  return `${ARCHIVE_ROOT}/${catalog.publicationId}/${csvFiles[0].filename}`;\n}\n\nasync function fetchAndParseCsv() {\n  // Discovery and archive download share the existing total request budget.\n  const signal = AbortSignal.timeout(120_000);\n  const catalogResponse = await fetch(CATALOG_URL, {\n    headers: { 'User-Agent': CHROME_UA },\n    signal,\n    redirect: 'error',\n  });\n  if (!catalogResponse.ok) throw new Error(`JODI Gas catalog fetch failed: HTTP ${catalogResponse.status}`);\n  const zipUrl = resolveGasArchiveUrl(await catalogResponse.json());\n  console.log(`  Fetching JODI Gas ZIP from ${zipUrl}`);\n  const resp = await fetch(zipUrl, {\n    headers: { 'User-Agent': CHROME_UA, 'Accept-Encoding': 'identity' },\n    signal,\n    redirect: 'error',\n  });\n  if (!resp.ok) throw new Error(`JODI Gas ZIP fetch failed: HTTP ${resp.status}`);\n\n  const arrayBuf = await resp.arrayBuffer();\n  const zipBuf = Buffer.from(arrayBuf);\n  console.log(`  ZIP downloaded: ${(zipBuf.length / 1024 / 1024).toFixed(1)} MB`);\n\n  const entry = findZipEntry(zipBuf, CSV_FILENAME);\n  if (!entry) throw new Error(`Could not find ${CSV_FILENAME} in ZIP`);\n  console.log(`  Found entry: ${entry.entryName} (compression=${entry.compression})`);\n\n  const compressed = zipBuf.slice(entry.dataOffset, entry.dataOffset + entry.compSize);","sourceCodeStart":266,"sourceCodeEnd":302,"githubUrl":"https://github.com/koala73/worldmonitor/blob/e586b8b4b80f595aa7ece295eec10d76f2921240/scripts/seed-jodi-gas.mjs#L266-L302","documentation":"fetchAndParseCsv() fetches the JODI Gas catalog JSON with a 120s timeout, a browser User-Agent, and redirects disabled. If the response is not ok (any 3xx/4xx/5xx after no-redirect enforcement), it throws 'JODI Gas catalog fetch failed: HTTP <status>'. This surfaces upstream availability problems with the exact status code.","triggerScenarios":"The catalog URL returns 403 (bot/CDN blocking the request), 404 (URL moved), 5xx (server outage), or 3xx (redirect: 'error' aborts on redirect, surfacing the 301/302 status).","commonSituations":"JODI moves the catalog endpoint without updating CATALOG_URL; a CDN/WAF blocks the fetcher's User-Agent; temporary upstream outage; the catalog starts redirecting to a new domain.","solutions":["curl the CATALOG_URL with the same User-Agent to reproduce and see the status/body","Update CATALOG_URL if JODI moved the endpoint (follow the redirect manually to find the new location)","Adjust headers (User-Agent/Accept) if a CDN is blocking, keeping the required server User-Agent policy","Retry later on 5xx if it's a transient outage; add retry/backoff around the fetch"],"exampleFix":"// before\nif (!catalogResponse.ok) throw new Error(`JODI Gas catalog fetch failed: HTTP ${catalogResponse.status}`);\n// after\nif (!catalogResponse.ok) throw new Error(`JODI Gas catalog fetch failed: HTTP ${catalogResponse.status} from ${CATALOG_URL}`);","handlingStrategy":"retry","validationCode":"const probe = await fetch(CATALOG_URL, { headers: { 'User-Agent': CHROME_UA }, redirect: 'error', signal: AbortSignal.timeout(120_000) });\nif (!probe.ok) console.warn(`catalog endpoint unhealthy: HTTP ${probe.status}`);","typeGuard":null,"tryCatchPattern":"try {\n  const catalogResponse = await fetch(CATALOG_URL, { headers: { 'User-Agent': CHROME_UA }, signal, redirect: 'error' });\n  if (!catalogResponse.ok) throw new Error(`JODI Gas catalog fetch failed: HTTP ${catalogResponse.status}`);\n} catch (err) {\n  if (err.name === 'TimeoutError') console.error('catalog fetch timed out after 120s');\n  else if (/HTTP \\d{3}/.test(err.message)) console.error(`${err.message}; check CATALOG_URL and CDN blocking`);\n  throw err;\n}","preventionTips":["Monitor the catalog URL's availability and status codes","Keep CATALOG_URL updated when JODI moves the endpoint","Send a browser-like User-Agent to avoid CDN/WAF bot blocking","Add retry with backoff for transient 5xx responses"],"tags":["network","http","external-api","data-seeding"],"backgroundTag":"http-error-response","analyzedSha":"e586b8b4b80f595aa7ece295eec10d76f2921240","analyzedAt":"2026-09-22T01:50:49.965Z","contentChangedAt":"2026-09-22T01:50:49.965Z","schemaVersion":2},"datasetVersion":"2026-09-23T08:17:48.524Z"}