JODI Gas catalog fetch failed: HTTP

Error message

JODI Gas catalog fetch failed: HTTP ${catalogResponse.status}

What it means

fetchAndParseCsv() fetches the JODI Gas catalog JSON with a 120s timeout, a browser User-Agent, and redirects disabled. If the response is not ok (any 3xx/4xx/5xx after no-redirect enforcement), it throws 'JODI Gas catalog fetch failed: HTTP <status>'. This surfaces upstream availability problems with the exact status code.

Solutions

curl the CATALOG_URL with the same User-Agent to reproduce and see the status/body
Update CATALOG_URL if JODI moved the endpoint (follow the redirect manually to find the new location)
Adjust headers (User-Agent/Accept) if a CDN is blocking, keeping the required server User-Agent policy
Retry later on 5xx if it's a transient outage; add retry/backoff around the fetch

Example fix

// before
if (!catalogResponse.ok) throw new Error(`JODI Gas catalog fetch failed: HTTP ${catalogResponse.status}`);
// after
if (!catalogResponse.ok) throw new Error(`JODI Gas catalog fetch failed: HTTP ${catalogResponse.status} from ${CATALOG_URL}`);

Defensive patterns

Strategy: retry

Validate before calling

const probe = await fetch(CATALOG_URL, { headers: { 'User-Agent': CHROME_UA }, redirect: 'error', signal: AbortSignal.timeout(120_000) });
if (!probe.ok) console.warn(`catalog endpoint unhealthy: HTTP ${probe.status}`);

Try / catch

try {
  const catalogResponse = await fetch(CATALOG_URL, { headers: { 'User-Agent': CHROME_UA }, signal, redirect: 'error' });
  if (!catalogResponse.ok) throw new Error(`JODI Gas catalog fetch failed: HTTP ${catalogResponse.status}`);
} catch (err) {
  if (err.name === 'TimeoutError') console.error('catalog fetch timed out after 120s');
  else if (/HTTP \d{3}/.test(err.message)) console.error(`${err.message}; check CATALOG_URL and CDN blocking`);
  throw err;
}

Prevention

Monitor the catalog URL's availability and status codes
Keep CATALOG_URL updated when JODI moves the endpoint
Send a browser-like User-Agent to avoid CDN/WAF bot blocking
Add retry with backoff for transient 5xx responses

When it happens

Trigger: The catalog URL returns 403 (bot/CDN blocking the request), 404 (URL moved), 5xx (server outage), or 3xx (redirect: 'error' aborts on redirect, surfacing the 301/302 status).

Common situations: JODI moves the catalog endpoint without updating CATALOG_URL; a CDN/WAF blocks the fetcher's User-Agent; temporary upstream outage; the catalog starts redirecting to a new domain.

Understand the failure class

Background: "API error: {status}" and "HTTP 401/403/404/429/5xx" errors: non-2xx HTTP responses explained — this error's family across 27 libraries.

Related errors

AI-assisted analysis of koala73/worldmonitor@e586b8b4b8 (2026-09-22). Data as JSON: /api/errors/3b75b1ed1a7a2627. Report an issue: GitHub.

Appendix: source

Thrown at scripts/seed-jodi-gas.mjs:284

    || !Array.isArray(catalog.files)) {
    throw new Error('JODI Gas catalog has no valid publication');
  }
  const csvFiles = catalog.files.filter(file => file?.format === 'CSV' && file.ignore === false);
  if (csvFiles.length !== 1 || csvFiles[0].filename !== 'GAS_world_NewFormat.zip') {
    throw new Error('JODI Gas catalog must contain one supported CSV archive');
  }
  return `${ARCHIVE_ROOT}/${catalog.publicationId}/${csvFiles[0].filename}`;
}

async function fetchAndParseCsv() {
  // Discovery and archive download share the existing total request budget.
  const signal = AbortSignal.timeout(120_000);
  const catalogResponse = await fetch(CATALOG_URL, {
    headers: { 'User-Agent': CHROME_UA },
    signal,
    redirect: 'error',
  });
  if (!catalogResponse.ok) throw new Error(`JODI Gas catalog fetch failed: HTTP ${catalogResponse.status}`);
  const zipUrl = resolveGasArchiveUrl(await catalogResponse.json());
  console.log(`  Fetching JODI Gas ZIP from ${zipUrl}`);
  const resp = await fetch(zipUrl, {
    headers: { 'User-Agent': CHROME_UA, 'Accept-Encoding': 'identity' },
    signal,
    redirect: 'error',
  });
  if (!resp.ok) throw new Error(`JODI Gas ZIP fetch failed: HTTP ${resp.status}`);

  const arrayBuf = await resp.arrayBuffer();
  const zipBuf = Buffer.from(arrayBuf);
  console.log(`  ZIP downloaded: ${(zipBuf.length / 1024 / 1024).toFixed(1)} MB`);

  const entry = findZipEntry(zipBuf, CSV_FILENAME);
  if (!entry) throw new Error(`Could not find ${CSV_FILENAME} in ZIP`);
  console.log(`  Found entry: ${entry.entryName} (compression=${entry.compression})`);

  const compressed = zipBuf.slice(entry.dataOffset, entry.dataOffset + entry.compSize);

View on GitHub (pinned to e586b8b4b8)