{"record":{"id":"373851ddf1e3e5f4","repo":"jackwener/OpenCLI","slug":"archive-snapshots-returned-malformed-json-error","errorCode":null,"errorMessage":"archive snapshots returned malformed JSON: ${error?.message || error}","messagePattern":"archive snapshots returned malformed JSON: (.+?)","errorType":"exception","errorClass":"CommandExecutionError","httpStatus":null,"severity":"error","filePath":"clis/archive/snapshots.js","lineNumber":85,"sourceCode":"        let resp;\n        try {\n            resp = await fetch(apiUrl, {\n                headers: {\n                    'Accept': 'application/json',\n                    'User-Agent': 'opencli/1.0 (+https://github.com/jackwener/opencli)',\n                },\n            });\n        } catch (error) {\n            throw new CommandExecutionError(`archive snapshots request failed: ${error?.message || error}`);\n        }\n        if (!resp.ok) {\n            throw new CommandExecutionError(`archive snapshots failed: HTTP ${resp.status}`);\n        }\n        let data;\n        try {\n            data = await resp.json();\n        } catch (error) {\n            throw new CommandExecutionError(`archive snapshots returned malformed JSON: ${error?.message || error}`);\n        }\n\n        // CDX returns an array of arrays; the first row is the header.\n        if (!Array.isArray(data)) {\n            throw new CommandExecutionError('archive snapshots returned malformed CDX payload: top-level payload must be an array');\n        }\n        if (data.length < 2) {\n            throw new EmptyResultError('archive snapshots', `No Wayback snapshots for \"${target}\".`);\n        }\n        const [header, ...rows] = data;\n        if (!Array.isArray(header)) {\n            throw new CommandExecutionError('archive snapshots returned malformed CDX payload: header row must be an array');\n        }\n        const cols = {};\n        header.forEach((name, i) => { cols[name] = i; });\n        const timestampCol = requireCdxColumn(cols, 'timestamp');\n        const originalCol = requireCdxColumn(cols, 'original');\n        const statusCol = requireCdxColumn(cols, 'statuscode');","sourceCodeStart":67,"sourceCodeEnd":103,"githubUrl":"https://github.com/jackwener/OpenCLI/blob/49907e53dc3ade5c223ff0c4c2c2785687cec4e6/clis/archive/snapshots.js#L67-L103","documentation":"The `archive snapshots` command queries the Wayback Machine CDX API (http://web.archive.org/cdx/search/cdx) and parses the response with `resp.json()`. This error is thrown when the HTTP response body cannot be parsed as JSON at all, wrapping the underlying parser message. The library throws it because the CDX API is expected to always return valid JSON when `output=json` is requested, so a parse failure means the endpoint returned something else (HTML error page, proxy intercept, empty body).","triggerScenarios":"The fetch to the CDX endpoint succeeded with an HTTP 2xx status, but `await resp.json()` threw — e.g. the server returned an HTML maintenance/interstitial page with status 200, a captive portal or corporate proxy replaced the body, the connection was truncated mid-body, or an intermediate cache returned a non-JSON error page.","commonSituations":"Corporate proxies or firewalls rewriting the web.archive.org response; archive.org serving an HTML status page during outages while still returning 200; node environments without proper network egress where a middleware injects content; intermittent network failures truncating the response stream.","solutions":["Re-run the command; the failure is often transient (proxy or archive.org hiccup).","Check network egress/proxy settings — ensure no captive portal or SSL-inspecting proxy is rewriting web.archive.org responses (try `curl 'http://web.archive.org/cdx/search/cdx?url=example.com&output=json&limit=2'`).","Call the CDX endpoint over HTTPS (`https://web.archive.org/cdx/search/cdx`) manually and inspect the raw body to see what is actually being returned.","If archive.org is down (check status.archive.org), wait and retry later."],"exampleFix":"// before: relying on the CLI directly behind a rewriting proxy\nawait exec('opencli archive snapshots example.com');\n// after: verify the endpoint returns JSON first, then call\nconst probe = await fetch('http://web.archive.org/cdx/search/cdx?url=example.com&output=json&limit=1');\nconst text = await probe.text();\nif (text.trim().startsWith('[')) { await exec('opencli archive snapshots example.com'); }","handlingStrategy":"try-catch","validationCode":"// Optional pre-check that the CDX endpoint currently serves JSON\nconst probe = await fetch('http://web.archive.org/cdx/search/cdx?url=example.com&output=json&limit=1');\nconst head = (await probe.text()).trim();\nif (!head.startsWith('[')) console.warn('CDX endpoint not returning JSON right now');","typeGuard":null,"tryCatchPattern":"try {\n  await exec('opencli archive snapshots example.com');\n} catch (e) {\n  if (String(e.message).includes('returned malformed JSON')) {\n    // treat as transient upstream/proxy issue: log raw error, retry or fall back\n  } else throw e;\n}","preventionTips":["Check status.archive.org before bulk runs","Verify proxy/VPN is not intercepting web.archive.org traffic","Retry transient failures with backoff","Probe the endpoint with curl when on new networks"],"tags":["network","json","http","wayback-machine"],"backgroundTag":"malformed-json-response","analyzedSha":"49907e53dc3ade5c223ff0c4c2c2785687cec4e6","analyzedAt":"2026-08-29T08:14:47.543Z","schemaVersion":2},"datasetVersion":"2026-08-29T12:17:43.993Z"}