jackwener/OpenCLI · error · CommandExecutionError

archive snapshots returned malformed CDX payload: snapshot r

Error message

archive snapshots returned malformed CDX payload: snapshot row is missing timestamp/original URL

What it means

Each snapshot row must contain a 14-digit `timestamp` (YYYYMMDDhhmmss) and a non-empty `original` URL. This error is thrown when either cell is absent, empty, or the timestamp fails the `/^\d{14}$/` check, since the library uses them to construct snapshot URLs.

Source

Thrown at clis/archive/snapshots.js:115

            throw new CommandExecutionError('archive snapshots returned malformed CDX payload: header row must be an array');
        }
        const cols = {};
        header.forEach((name, i) => { cols[name] = i; });
        const timestampCol = requireCdxColumn(cols, 'timestamp');
        const originalCol = requireCdxColumn(cols, 'original');
        const statusCol = requireCdxColumn(cols, 'statuscode');
        const mimetypeCol = requireCdxColumn(cols, 'mimetype');

        return rows.slice(0, limit).map(row => {
            if (!Array.isArray(row)) {
                throw new CommandExecutionError('archive snapshots returned malformed CDX payload: snapshot row must be an array');
            }
            const timestamp = String(row[timestampCol] ?? '');
            const original = String(row[originalCol] ?? '');
            const status = row[statusCol];
            const mimetype = row[mimetypeCol];
            if (!/^\d{14}$/.test(timestamp) || !original) {
                throw new CommandExecutionError('archive snapshots returned malformed CDX payload: snapshot row is missing timestamp/original URL');
            }
            if (status == null || mimetype == null || String(status) === '' || String(mimetype) === '') {
                throw new CommandExecutionError('archive snapshots returned malformed CDX payload: snapshot row is missing statuscode/mimetype');
            }
            return {
                timestamp,
                snapshot_url: buildWaybackUrl(timestamp, original),
                status: String(status),
                mimetype: String(mimetype),
                original_url: original,
            };
        });
    },
});

View on GitHub (pinned to 49907e53dc)

Solutions

Re-run the command to see if the bad row is transient.
Inspect the raw CDX output for the offending row and note its timestamp/original values.
Narrow the query (specific path, tighter from/to, smaller limit) to skip malformed rows.
Update opencli or report the payload upstream if CDX changed its field semantics.

Defensive patterns

Strategy: validation

Validate before calling

// Verify a raw CDX row has the required cells before processing
function rowHasTimestampAndOriginal(row, tsCol, origCol) {
  return /^\d{14}$/.test(String(row[tsCol] ?? '')) && Boolean(row[origCol]);
}

Type guard

function hasValidTimestamp(v) {
  return typeof v === 'string' && /^\d{14}$/.test(v);
}

Try / catch

try {
  await exec('opencli archive snapshots example.com --limit 20');
} catch (e) {
  if (String(e.message).includes('missing timestamp/original URL')) {
    console.error('CDX returned a capture with blank timestamp/URL — narrow the query');
  } else throw e;
}

Prevention

Avoid CDX collapsing/dedup flags that produce truncated timestamps
Prefer exact-path queries over wildcard domain queries
Re-run once for transient ingest artifacts
Check raw payload when errors repeat for the same row

When it happens

Trigger: A CDX row has an empty/short timestamp (e.g. collapsed or truncated fields) or a missing `original` column value — typically from `collapse=` style server-side deduplication artifacts, truncated responses, or CDX API changes to field contents.

Common situations: Rows with collapsed timestamps of reduced precision; captures whose original URL was redacted/empty; proxy responses cutting cells; querying with flags that trigger CDX field collapsing.

Understand the failure class

Parsing and encoding errors: unexpected token, malformed input — why parsers reject input and how to find the real culprit.

Related errors

AI-assisted analysis of jackwener/OpenCLI@49907e53dc (2026-08-29). Data as JSON: /api/errors/3995eb901c082bf5. Report an issue: GitHub.