jackwener/OpenCLI · error · CommandExecutionError

archive snapshots returned malformed JSON: ${error?.message

Error message

archive snapshots returned malformed JSON: ${error?.message || error}

What it means

The `archive snapshots` command queries the Wayback Machine CDX API (http://web.archive.org/cdx/search/cdx) and parses the response with `resp.json()`. This error is thrown when the HTTP response body cannot be parsed as JSON at all, wrapping the underlying parser message. The library throws it because the CDX API is expected to always return valid JSON when `output=json` is requested, so a parse failure means the endpoint returned something else (HTML error page, proxy intercept, empty body).

Source

Thrown at clis/archive/snapshots.js:85

        let resp;
        try {
            resp = await fetch(apiUrl, {
                headers: {
                    'Accept': 'application/json',
                    'User-Agent': 'opencli/1.0 (+https://github.com/jackwener/opencli)',
                },
            });
        } catch (error) {
            throw new CommandExecutionError(`archive snapshots request failed: ${error?.message || error}`);
        }
        if (!resp.ok) {
            throw new CommandExecutionError(`archive snapshots failed: HTTP ${resp.status}`);
        }
        let data;
        try {
            data = await resp.json();
        } catch (error) {
            throw new CommandExecutionError(`archive snapshots returned malformed JSON: ${error?.message || error}`);
        }

        // CDX returns an array of arrays; the first row is the header.
        if (!Array.isArray(data)) {
            throw new CommandExecutionError('archive snapshots returned malformed CDX payload: top-level payload must be an array');
        }
        if (data.length < 2) {
            throw new EmptyResultError('archive snapshots', `No Wayback snapshots for "${target}".`);
        }
        const [header, ...rows] = data;
        if (!Array.isArray(header)) {
            throw new CommandExecutionError('archive snapshots returned malformed CDX payload: header row must be an array');
        }
        const cols = {};
        header.forEach((name, i) => { cols[name] = i; });
        const timestampCol = requireCdxColumn(cols, 'timestamp');
        const originalCol = requireCdxColumn(cols, 'original');
        const statusCol = requireCdxColumn(cols, 'statuscode');

View on GitHub (pinned to 49907e53dc)

Solutions

  1. Re-run the command; the failure is often transient (proxy or archive.org hiccup).
  2. Check network egress/proxy settings — ensure no captive portal or SSL-inspecting proxy is rewriting web.archive.org responses (try `curl 'http://web.archive.org/cdx/search/cdx?url=example.com&output=json&limit=2'`).
  3. Call the CDX endpoint over HTTPS (`https://web.archive.org/cdx/search/cdx`) manually and inspect the raw body to see what is actually being returned.
  4. If archive.org is down (check status.archive.org), wait and retry later.

Example fix

// before: relying on the CLI directly behind a rewriting proxy
await exec('opencli archive snapshots example.com');
// after: verify the endpoint returns JSON first, then call
const probe = await fetch('http://web.archive.org/cdx/search/cdx?url=example.com&output=json&limit=1');
const text = await probe.text();
if (text.trim().startsWith('[')) { await exec('opencli archive snapshots example.com'); }
Defensive patterns

Strategy: try-catch

Validate before calling

// Optional pre-check that the CDX endpoint currently serves JSON
const probe = await fetch('http://web.archive.org/cdx/search/cdx?url=example.com&output=json&limit=1');
const head = (await probe.text()).trim();
if (!head.startsWith('[')) console.warn('CDX endpoint not returning JSON right now');

Try / catch

try {
  await exec('opencli archive snapshots example.com');
} catch (e) {
  if (String(e.message).includes('returned malformed JSON')) {
    // treat as transient upstream/proxy issue: log raw error, retry or fall back
  } else throw e;
}

Prevention

When it happens

Trigger: The fetch to the CDX endpoint succeeded with an HTTP 2xx status, but `await resp.json()` threw — e.g. the server returned an HTML maintenance/interstitial page with status 200, a captive portal or corporate proxy replaced the body, the connection was truncated mid-body, or an intermediate cache returned a non-JSON error page.

Common situations: Corporate proxies or firewalls rewriting the web.archive.org response; archive.org serving an HTML status page during outages while still returning 200; node environments without proper network egress where a middleware injects content; intermittent network failures truncating the response stream.

Understand the failure class

Related errors

AI-assisted analysis of jackwener/OpenCLI@49907e53dc (2026-08-29). Data as JSON: /api/errors/373851ddf1e3e5f4. Report an issue: GitHub.