jackwener/OpenCLI · error · CommandExecutionError

archive snapshots returned malformed CDX payload: snapshot r

Error message

archive snapshots returned malformed CDX payload: snapshot row is missing statuscode/mimetype

What it means

Beyond timestamp and original URL, every snapshot row must have non-null, non-empty `statuscode` and `mimetype` cells. The library throws this when either is missing or blank, because these are required output columns and are coerced to strings for the result rows.

Source

Thrown at clis/archive/snapshots.js:118

        header.forEach((name, i) => { cols[name] = i; });
        const timestampCol = requireCdxColumn(cols, 'timestamp');
        const originalCol = requireCdxColumn(cols, 'original');
        const statusCol = requireCdxColumn(cols, 'statuscode');
        const mimetypeCol = requireCdxColumn(cols, 'mimetype');

        return rows.slice(0, limit).map(row => {
            if (!Array.isArray(row)) {
                throw new CommandExecutionError('archive snapshots returned malformed CDX payload: snapshot row must be an array');
            }
            const timestamp = String(row[timestampCol] ?? '');
            const original = String(row[originalCol] ?? '');
            const status = row[statusCol];
            const mimetype = row[mimetypeCol];
            if (!/^\d{14}$/.test(timestamp) || !original) {
                throw new CommandExecutionError('archive snapshots returned malformed CDX payload: snapshot row is missing timestamp/original URL');
            }
            if (status == null || mimetype == null || String(status) === '' || String(mimetype) === '') {
                throw new CommandExecutionError('archive snapshots returned malformed CDX payload: snapshot row is missing statuscode/mimetype');
            }
            return {
                timestamp,
                snapshot_url: buildWaybackUrl(timestamp, original),
                status: String(status),
                mimetype: String(mimetype),
                original_url: original,
            };
        });
    },
});

View on GitHub (pinned to 49907e53dc)

Solutions

Re-run the command; in-flight captures often gain their metadata shortly after.
Inspect the raw CDX output to identify rows with blank statuscode/mimetype.
Restrict the time range or path to exclude the problematic captures.
If it persists for all rows, check archive.org status and opencli updates.

Defensive patterns

Strategy: validation

Validate before calling

// Ensure statuscode/mimetype cells are populated before consuming results
function rowHasStatusAndMimetype(row, stCol, mtCol) {
  return row[stCol] != null && row[mtCol] != null && String(row[stCol]) !== '' && String(row[mtCol]) !== '';
}

Type guard

function isPopulated(v) {
  return v != null && String(v).trim() !== '';
}

Try / catch

try {
  await exec('opencli archive snapshots example.com --limit 20');
} catch (e) {
  if (String(e.message).includes('missing statuscode/mimetype')) {
    // likely a tombstoned/in-flight capture — retry later or narrow range
  } else throw e;
}

Prevention

Wait and retry — new captures often gain metadata shortly after ingest
Exclude robot-blocked periods via from/to ranges
Query exact paths to reduce tombstoned row exposure
Check status.archive.org if all rows fail

When it happens

Trigger: A CDX row where `statuscode` or `mimetype` is null or empty string — common for tombstoned captures (redirects to a block page), newly ingested captures with pending metadata, or rows produced by CDX server-side collapsing.

Common situations: Removed/robot-blocked captures that still appear in CDX but lack status/mimetype; captures in the middle of archive.org's ingest pipeline; querying very recent or very old edge-case captures.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of jackwener/OpenCLI@49907e53dc (2026-08-29). Data as JSON: /api/errors/34745dca825683c1. Report an issue: GitHub.